Apparatus for generating a coherent beam illumination

ABSTRACT

An apparatus generates a coherent illumination beam. An embedded light-scattering apparatus in a transparent substrate illuminates a reflective optical element which is also embedded inside the same substrate. The reflective optical element is designed to provide a desired beam profile.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a divisional application of U.S. patent application Ser. No.15/632,164 filed Jun. 23, 2017 by Hakan Urey et al., titled “NEAR-TO-EYEDISPLAY DEVICE,” which is a continuation of, and claims priority benefitunder 35 USC §365(c) of, PCT Patent Application No. PCT/TR2014/000512filed Dec. 26, 2014 by Hakan Urey et al., titled “NEAR-TO-EYE DISPLAYDEVICE,” PCT Patent Application No. PCT/TR2014/000513 filed Dec. 26,2014 by Hakan Urey, titled “NEAR-TO-EYE DISPLAY DEVICE WITH SPATIALLIGHT MODULATOR AND PUPIL TRACKER,” PCT Patent Application No.PCT/TR2014/000514 filed Dec. 26, 2014 by Hakan Urey et al., titled“NEAR-TO-EYE DISPLAY DEVICE WITH MOVING LIGHT SOURCES,” PCT PatentApplication No. PCT/TR2014/000515 filed Dec. 26, 2014 by Hakan Urey etal., titled “APPARATUS FOR GENERATING A COHERENT BEAM ILLUMINATION,” andPCT Patent Application No. PCT/TR2014/000516 filed Dec. 26, 2014 byHakan Urey et al., titled “NEAR-TO-EYE DISPLAY DEVICE WITH VARIABLERESOLUTION,” each of which is incorporated herein by reference in itsentirety.

FIELD OF THE INVENTION

The present invention relates generally to optical systems, and morespecifically to near-to-eye display devices.

BACKGROUND

Head-worn displays (HWD) typically employ a microdisplay on which atwo-dimensional (2D) regular image is displayed. Since the physicaldistance between the microdisplay and the eye is typically much smallerthan 25 cm (the closest distance at which the human eye can normallyfocus), a blurred image forms on the retina unless relay optics areplaced in between. The relay optics typically consist of several lenseswhich serve to form a magnified virtual image of the microdisplay beyond25 cm (mostly at infinity) on which the eye can then focus and form asharp retinal image.

Lightweight HWD designs that employ microdisplays (those that use only asingle magnifier lens, for instance) are mostly restricted to systemshaving small fields of view (FOV), since weight and bulk increase forlarge FOV designs due to additional components inserted to compensatefor aberrations. As an example, the recently emerging Google Glass(which has a quite thin form factor) basically consists of a small(˜1-cm diagonal) microdisplay and a simple positive lens, but has alimited FOV, beyond which aberrations become severe. On the other hand,high-end military-type displays may support an FOV approaching 150degrees or more, but weigh as much as 5 kg or more and may contain morethan 10 different lenses, most of which are present to compensate foraberrations that emerge due to the enlarged FOV. Having so many lensesis not merely a technological problem, but a fundamental one, since nosingle optical component can be designed to form an aberration freeimage of a large size microdisplay, due to the fact that the informationemerging from the microdisplay quickly gets spread in space as itpropagates.

Microdisplay-based HWD designs also fall short of providing the ultimatethree-dimensional (3D) visual experience. These HWD designs typicallyprovide only stereoscopic images, which invoke 3D perception essentiallyonly through binocular disparity. Monocular cues, especiallyaccommodation, are typically not supported, or are incorrect. Users ofstereoscopic systems typically suffer from visual fatigue caused by theso-called accommodation-convergence conflict, in which eyes convergetruly to the apparent position of a 3D object while accommodation is setincorrectly to the screen so as to make retinal images sharp. Thefatigue is especially severe when virtual objects are closer than 50 cm.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a perspective view of a near-to-eye display device;

FIG. 2 shows a top view of the near-to-eye display device of FIG. 1;

FIG. 3 shows a handheld near-to-eye display device;

FIG. 4 shows a cross section of a spatial light modulator (SLM) beingilluminated and generating a virtual-scene wave;

FIG. 5 shows the cross section of FIG. 4 depicting the virtual scene asseen by a user;

FIG. 6 shows a spatial light modulator with a pixelated structure;

FIG. 7 shows a cross section of an SLM that generates noise beams andmultiple diffraction orders;

FIG. 8 shows the cross section of FIG. 7 with a user's eye pupilfiltering out unwanted noise beams and diffraction orders;

FIGS. 9, 10, and 11 show multiple diffraction orders on an exit pupilplane with a useful portion;

FIG. 12 shows an optical architecture in which the SLM is placed on aconverging beam path;

FIG. 13 shows an optical architecture in which the SLM is illuminated bya diverging wavefront;

FIG. 14 shows an optical architecture with a point light source and SLM,with no other components with refractive power;

FIG. 15 shows an optical architecture in which an SLM is illuminated ina time sequential manner by an array of point light sources;

FIG. 16 shows an optical architecture with multiple light sources andapertures to the associated emission cones;

FIG. 17 shows an optical architecture in which a reflective SLM isplaced directly in front of the user's eye;

FIGS. 18, 19, and 20 show optical architectures in which real-worldvision is not blocked by the SLM;

FIG. 21 shows an optical architecture in which a reflective SLM isplaced in front of the user's eye;

FIGS. 22-28 show optical architectures in which real-world vision is notblocked by the SLM;

FIG. 29 shows an optical architecture in which an SLM and reflector arecombined;

FIG. 30 shows a reflector based solution for modulation in a singledirection;

FIG. 31 shows a virtual-reality (VR) architecture with SLM tiling;

FIG. 32 shows a VR architecture with SLM tiling;

FIG. 33 shows a VR display architecture;

FIG. 34 shows two tiled SLMs to double resolution;

FIG. 35 shows a near-to-eye display device with a high-resolutionportion and a low-resolution portion;

FIG. 36 shows a high-resolution image for foveal vision and lowerresolution for peripheral vision;

FIG. 37 shows the high-resolution image being steered to a user's pupilposition;

FIGS. 38 and 39 show a display system with a rotating hologram module tocreate a steerable high-resolution image;

FIG. 40 shows a portion of a near-to-eye display device having a movingplatform;

FIG. 41 shows a moving platform upon which an SLM is mounted;

FIG. 42 shows a portion of a near-to-eye display device having a movingplatform with an array of bars;

FIG. 43 shows a moving platform having an array of bars upon which SLMsand microdisplays are mounted;

FIG. 44 shows a moving platform that moves in two dimensions to increaseresolution;

FIG. 45 shows a near-to-eye display device with a transducer to interactwith a user for calibration;

FIG. 46 shows a flowchart of calibration methods in accordance withvarious embodiments of the invention;

FIG. 47 shows example images shown to a user during calibration;

FIG. 48 shows a flowchart of calibration methods in accordance withvarious embodiments of the invention;

FIG. 49 shows example images shown to a user during calibration;

FIG. 50 shows a flowchart of calibration methods in accordance withvarious embodiments of the invention;

FIG. 51 shows example images shown to a user during calibration;

FIG. 52 shows a flowchart of calibration methods in accordance withvarious embodiments of the invention;

FIG. 53 shows a near-to-eye display device with actuators forcalibration;

FIG. 54 shows images of a user's eyes used for calibration;

FIG. 55 shows the near-to-eye display device of FIG. 53 with actuationfor calibration;

FIG. 56 shows a flowchart representing computation of SLM data;

FIG. 57-64 show a number of space-angle (or space-frequency)distributions that illustrate the basics of the computation procedure;

FIG. 65 shows fast hologram computation;

FIG. 66 illustrates the fundamentals of the method for deliveringspeckle-free images to the retina of a user;

FIG. 67 shows a perspective drawing of a back-light unit that generatesa two-dimensional converging beam;

FIG. 68 shows a cross section of the back-light unit of FIG. 67 showinga scattering point and linearly arranged micromirror array;

FIG. 69 shows a cross section of the back-light unit of FIG. 67 showinga light-scattering apparatus and a reflective optical element arrangedas a Fresnel mirror;

FIG. 70 shows a cross section of the back-light unit of FIG. 67 showinga light-scattering apparatus and a reflective optical element arrangedas a free form concave reflector;

FIG. 71 shows a cross section of the back-light unit of FIG. 67 showinga scattering point and nonlinearly arranged micromirror array;

FIG. 72 shows a back-light unit with an external light source;

FIG. 73 shows a cross section of a back-light unit with transmissiveSLM;

FIG. 74 shows a cross section of a back-light unit with a reflectiveSLM;

FIG. 75 shows a cross section of back-light unit with cross polarizers;

FIG. 76 shows a cross section of back-light unit with a mirror;

FIG. 77 shows a cross section of a back-light unit with a fiber;

FIG. 78 shows a perspective view of a back-light unit that generatesone-dimensional converging beam;

FIG. 79 shows a perspective view of a back-light unit that generates acollimated beam;

FIG. 80 shows a perspective view of a back-light unit that generates adiverging beam;

FIG. 81 shows a cross section of a slab waveguide, a wedge, and acomponent with a micromirror array;

FIG. 82 shows a top view of the apparatus of FIG. 81;

FIG. 83 shows a cross section of a slab, wedge, component withmicromirror array, and SLM positioned along the slab;

FIG. 84 shows a cross section of a slab waveguide, a wedge, a componentwith a micromirror array, and an SLM between the wedge and the componentwith the micromirror array;

FIG. 85 shows a cross section of slab waveguide, wedge, component with amicromirror array, and an SLM below the wedge;

FIG. 86 shows a cross section of a slab waveguide, wedge, component withmicromirror array, and an SLM at entrance to the slab;

FIG. 87 shows a cross section of a slab waveguide, wedge, compensatingwedge with micromirror array, and SLM below the wedge;

FIG. 88 shows a cross section of a slab waveguide with a 90-degree bend,wedge, optical component with a micromirror array, and an SLM;

FIG. 89 shows a cross section of a slab waveguide, wedge, and camera foreye tracking;

FIG. 90 shows a near-to-eye display device with a slab waveguide, wedge,component with micromirror array, SLM, and camera for eye tracking;

FIG. 91 shows a slab waveguide, a curved wedge, and a compensationplate;

FIG. 92 shows a slab waveguide, curved wedge, and SLM in a convergingbeam;

FIG. 93 shows a slab waveguide, curved wedge, and SLM on top of theslab;

FIG. 94 shows a slab waveguide, curved wedge, and SLM at the entrance tothe slab waveguide;

FIG. 95 shows a slab waveguide, curved wedge, and camera for eyetracking;

FIG. 96 shows a perspective view of the apparatus of FIG. 91;

FIG. 97 shows a near-to-eye display device with a slab waveguide, curvedwedge, SLM, and camera for eye tracking;

FIG. 98 shows a near-to-eye display device with a moving platformassembly;

FIG. 99 shows a cross section of a moving platform assembly;

FIG. 100 shows a perspective view of a moving platform assembly;

FIG. 101 shows a side view of contact lens placed on an eye;

FIG. 102 shows a front view of the contact lens of FIG. 101;

FIG. 103 shows a cross section of a contact lens on an eye and a movingplatform assembly;

FIG. 104 shows a near-to-eye display device with a moving platformassembly;

FIG. 105 shows a perspective view of a near-to-eye display device with arotating bar;

FIGS. 106-108 show front views of near-to-eye display devices withrotating bars;

FIGS. 109 and 110 show rotating-bar-actuation embodiments;

FIG. 111 shows a front view of a near-to-eye display device with aplatform that moves in two dimensions;

FIG. 112 shows an external display with no contact lens;

FIG. 113 shows a perspective view of near-to-eye display device thatincludes an LED array.

FIG. 114 shows a two-dimensional LED array;

FIGS. 115 and 116 show a top view of pupil tracking using multiple LEDs;

FIG. 117 shows a near-to-eye display device that includes a rotatingSLM.

FIGS. 118 and 119 show a top view of pupil tracking using a rotatingSLM;

FIG. 120 shows a perspective view of a near-to-eye display device thatincludes rotating SLMs and LED arrays.

FIG. 121 shows a flowchart showing rotation for small angles and LEDselection for larger angles;

FIG. 122 shows a flowchart showing rotation for small angles anddiffraction order selection for larger angles;

FIG. 123 shows a near-to-eye display device that includes activegrating;

FIGS. 124 and 125 show top views of pupil tracking using an SLM and anactive grating;

FIG. 126 shows a near-to-eye display device with a combination of anactive grating and an LED array;

FIG. 127 shows a flowchart showing grating actuation for small anglesand LED selection for larger angles;

FIG. 128 shows a flowchart showing grating actuation for small anglesand diffraction order selection for larger angles;

FIGS. 129 and 130 show augmented-reality views demonstrating a virtualscene at different depths;

FIG. 131 shows a block diagram of a near-to-eye display device inaccordance with various embodiments of the present invention; and

FIG. 132 shows a near-to-eye display device with transparenttouch-sensitive layers.

DESCRIPTION OF EMBODIMENTS

In the following detailed description, reference is made to theaccompanying drawings that show, by way of illustration, specificembodiments in which the invention may be practiced. These embodimentsare described in sufficient detail to enable those skilled in the art topractice the invention. It is to be understood that the variousembodiments of the invention, although different, are not necessarilymutually exclusive. For example, a particular feature, structure, orcharacteristic described herein in connection with one embodiment may beimplemented within other embodiments without departing from the scope ofthe invention. In addition, it is to be understood that the location orarrangement of individual elements within each disclosed embodiment maybe modified without departing from the scope of the invention. Thefollowing detailed description is, therefore, not to be taken in alimiting sense, and the scope of the present invention is defined onlyby the appended claims, appropriately interpreted, along with the fullrange of equivalents to which the claims are entitled. In the drawings,like numerals refer to the same or similar functionality throughout theseveral views.

FIG. 1 shows a perspective view of a near-to-eye display device.Near-to-eye display device 100 includes a frame 102 in the shape of aneyeglass frame. Near-to-eye display device 100 also includes spatiallight modulators (SLM) 110, point light source 120, electronics module160, and cable 170. In embodiments represented by FIG. 1, thenear-to-eye display device is a head-worn device (HWD), although this isnot a limitation of the present invention. In some embodiments,near-to-eye display devices are not head-worn. Various examples ofnon-head-worn near-to-eye display devices are discussed further below.

Point light source 120 may include any type of light source. Forexample, in some embodiments, point light source 120 includes a laserlight source. Also for example, in some embodiments, point light source120 includes an ultraviolet (UV) light source, an infrared (IR) lightsource, or other source of visible or nonvisible light.

In operation, near-to-eye display device 100 displays a monochrome orfull-color video of a 2D or 3D virtual scene 150 that appears to belocated on the outer side of the eyeglass to the user. For each frame ofthe displayed video, point light source 120 generates a coherent lightwave of a single wavelength that illuminates a spatial light modulator(SLM 110) that is mounted on the front section of the eyeglass. The SLMspatially modulates the phase and/or amplitude of the incident wave andreflects it towards the eye of the user, shown generally at 130. In someembodiments, near-to-eye display device 100 is a monochrome displaydevice, and point light source 120 only generates a coherent light waveof a single color. In other embodiments, near-to-eye display device 100is a full-color display device, and point light source 120 generatescoherent light waves of different wavelengths in a time sequentialmanner.

For each video frame, the data on the SLM is a computer-generatedholographic image of the virtual scene. The data on the SLM is computedand fed by a computer unit, which can be mounted on frame 102 aselectronics module 160, or can be connected to the display device bycable 170 or wireless links (not shown).

Electronics module 160 may include any suitable components. For example,in some embodiments, electronics module 160 includes driver circuits todrive point light source 120, and digital processing components to storeSLM data and to drive the SLMs 110 with that data. Also for example,electronics module 160 may include a processor and memory, or any othersuitable electronics components.

In some embodiments, SLM data is computed real-time as it is displayed.In these embodiments, electronics module 160 computes the SLM data anddrives SLMs 110 with the SLM data to create virtual scene 150 inreal-time. The real-time SLM data may be a function of head-trackingdata, pupil-tracking data, environmental data (e.g., ambient light,objects in the user's field of view, etc.).

In other embodiments, SLM data is precomputed and stored for retrievalat display time. For example, SLM data for an entire virtual environmentmay be precomputed and stored. As a user traverses the virtualenvironment, the appropriate SLM data is retrieved and displayed. Instill further embodiments, portions of the SLM data are precomputed, andportions of the SLM data are computed real-time.

Point light source 120 is shown on an outer portion of frame 102, andSLM 110 is reflective. In other embodiments, point light source islocated differently, and the SLM is transmissive. For example, in someembodiments, point light source 120 is mounted on the nose bridgebetween the two SLMs, and the light generated by point light source 120is fed to a back-light unit that illuminates the SLM from the oppositeside than shown in FIG. 1. These and other optical architectures aredescribed below with reference to later figures.

FIG. 2 shows a top view of the near-to-eye display device of FIG. 1. Thewave 130 reflected by SLM 110 propagates towards the user's eye 210 andforms a light wave distribution on the exit pupil plane 220, which isdefined as the plane that lies just in front of the user's eye, andcorresponds to the expected location of the user's eye pupil 212. Partof the light wave distribution formed on the exit pupil plane isintercepted by the user's eye pupil 212 and propagates to the retina,where a 3D image of the virtual scene is formed. In some embodiments, areal-world view is superimposed on the virtual scene, and in otherembodiments, the real-world view is blocked, and the only image formedon the retina is the virtual scene.

In general, systems that display a virtual scene and block thereal-world view are referred to as “virtual reality” (VR) systems, andsystems that superimpose the real-world view with the virtual scene arereferred to as “augmented reality” (AR) systems.

FIG. 3 shows a handheld near-to-eye display device. As used herein, theterm “near-to-eye display device” refers to any device that produces alight wave distribution of a virtual scene on an exit pupil plane from aphysical distance that is less than the typical minimal distance atwhich the human eye can normally focus (e.g., 25 cm). A near-to-eyedisplay device may be handheld as in FIG. 3, or may head-worn as inFIG. 1. A near-to-eye display device may also be stationary forapplications in which a user is expected to place their head against ornear the near-to-eye display device (e.g., VR demonstrators). Theexample handheld near-to-eye display device of FIG. 3 is in the shape ofa pair of binoculars, but this is not a limitation of the presentinvention. Any type of near-to-eye display device: head-worn, handheld(e.g., electronic viewfinders in cameras, foldable image viewer,smartphones), or otherwise, may include embodiments of the presentinvention.

Any near-to-eye display device (e.g., near-to-eye display device 300)may include any of the invention embodiments described herein. Forexample, any near-to-eye display device may include any of theembodiments that produce a light wave distribution on the exit pupilplane 220 described herein.

FIG. 4 shows a cross section of an SLM being illuminated and generatinga virtual-scene wave. SLM 410 is shown as a transmissive SLM.Illumination optics module 440 produces, and illuminates SLM 410 with, acoherent illumination wave 442. SLM 410 modulates the light and createsvirtual-scene wave 412. Encoded in virtual-scene wave 412 is a 3Dvirtual scene that is imaged on the user's retina 424. Only the portionof the virtual-scene wave that intersects the user's pupil 212 on theexit pupil plane 220 creates an image on the retina. Other informationin the virtual-scene wave that falls outside the user's pupil isfiltered out and does not enter the user's eye. Various inventionembodiments that employ pupil filtering are discussed in more detailbelow.

Illumination optics module 440 shown here creates a convergingillumination wave. In some embodiments, this is accomplished with lightsources and optical components such as mirrors, micromirror arrays,lenses, and the like. Various embodiments of illumination optics modulesare described in more detail below. In some embodiments, theillumination optics module does not necessarily generate a convergingillumination wave. For example, one simple example of an illuminationoptics module is a point light source 120 (FIG. 1). In that case, theillumination wave is a diverging wave. Yet, in other embodiments shownbelow, the illumination wave is generated by arrays containing multiplepoint light sources. However, in any case, the illumination wave mustpossess a certain degree of spatial coherency over sufficiently largeareas of the SLM.

SLMs are basically dynamically programmable diffractive opticalelements. Various different SLM technologies exist. SLMs based onnematic liquid crystals (LC) make use of the electrically controlledrefractive index of anisotropic LCs to modulate polarization, intensityor phase of incident light. The type of modulation depends on the modeof the LC that is used. Twisted nematic LCs rotate the polarization ofincident light by some controlled amount, and are used along withpolarizers on both sides to constitute intensity modulators suitable forincoherent light applications, most commonly, 2D displays. Parallelaligned nematic (PAN) (or electrically controlled birefringence (ECB))mode LCs are most suitable for coherent light applications, and they canbe used as multilevel phase only SLMs. Transmissive SLMs based on LCshave large pixel pitch due to the fact that electronic circuitryassociated with each pixel must be embedded within the pixel aperture.Reflective SLMs based on Liquid Crystal on Silicon (LCoS) technology canbe made to have much smaller pixel pitches, since electronics can beburied under the pixel. One advantage of SLMs based on nematic LCs isthe multilevel modulation these devices can perform. However, theirperformance is limited by pixel crosstalk and low frame rates, which maybe problematic in color field sequential holographic applications. SLMsbased on ferroelectric LCs have much higher frame rates at the cost ofmerely binary modulation at each pixel.

Microelectromechanical systems (MEMS) based SLMs are advantageous interms of frame rate and basically no pixel crosstalk. DigitalMicromirror Device (DMD) can be used as SLM. However, it provides onlybinary modulation. Moreover, the complicated pixel structure of thesedevices makes it difficult to reduce the pixel pitch. Other MEMS SLMscan be implemented using deformable membrane structures, piston motionmicromirror arrays, programmable diffraction gratings such as gratinglight valve devices, electro-wetting and magneto-optic Kerr effectdevices, or MEMS laser scanners.

Various embodiments of the present invention may employ any of the SLMtechnologies discussed above, or others, including but not limited to:Optically Addressed SLMs, Acousto-Optical SLMs, Magneto Optical SLMs,MEMS mirrors, and the like.

FIG. 5 shows the cross section of FIG. 4 depicting the virtual scene asseen by a user. Virtual scene 150 includes one virtual object: a 3Drepresentation of a car. Any number of objects may be included in thevirtual scene without departing from the scope of the present invention.In operation, SLM 410 converts the illumination wave to thevirtual-scene wave that would be emanated by virtual scene 150.

The restrictions of SLMs have important implications on thecapabilities, limitations, and design of the various embodiments of theinvention. As explained above, and as shown in FIG. 4, in operation, theSLM is illuminated by a coherent wavefront, which is generated by agroup of optical components and light sources that are part ofillumination optics module 440. The computer-generated holographic imagedisplayed on the SLM helps convert the illumination wave to thevirtual-scene wave that would be emanated by virtual scene 150.Therefore, the SLM is the device where information about the virtualscene is fed to the light wave that is delivered to the eye of the user.However, due to the restrictions and limitations of real SLMs, the SLMis able to synthesize only a portion of the wave emanated by the virtualscene, and the incident wave is only partially converted to the waveemanated by the virtual scene.

In particular, real SLMs have finite spatial sizes, which restrict thesize of a virtual scene that is displayed (or, the field of view (FOV)within which the virtual scene is visible), and finite spatialbandwidths (pixel pitches usually several multiples of wavelength),which limits the portion of the wave emanating from each virtual-scenepoint that can be reconstructed.

SLMs also generate higher diffraction orders as a result of theirpixelated structure. These orders correspond to shifted replicas ofvirtual scenes that are to be displayed, which appear as ghost imagereplicas if they are allowed to enter the eye and propagate to theretina.

Further, computer-generated holographic images that perform theconversion of an illumination wave to waves emanated by the virtualscene are in general analog and complex-valued, whereas practical SLMscan only perform some restricted type of modulation: phase only,amplitude only, binary, etc., and do provide only a limited number ofdistinct values. Therefore, a restricted type computer-generatedholographic image, which quantizes and encodes the ideal complex-valuedcomputer-generated holographic image, is computed and displayed on theSLM. However, this procedure leads to the emergence of additionalundesired beams, that we refer to as “noise beams,” in addition to thedesired wave. The encoding should be performed such that the resultingnoise beams do not enter into the eye, because otherwise, significantbackground noise will be observed over displayed images.

In the case of SLMs that essentially provide real valued modulation,such as binary SLMs or amplitude only SLMs, a conjugate beam will beformed. This conjugate beam, which corresponds to the wave emitted by asecond virtual scene which is the mirror image of the actual virtualscene with respect to the exit pupil plane, should also be preventedfrom entering the eye.

Further, some SLMs leave a component of the incident wave unmodulated.This component, which we refer to as the unmodulated “DC beam,” shouldalso be prevented from entering the eye.

Computational methods for generating the holographic images to bedisplayed on the SLM are described further below with reference to laterfigures.

FIG. 6 shows an SLM with a pixelated structure. The pixelated structureof SLMs is intimately linked with sampling and interpolation of lightwaves. The final analog optical-mask structure that is implemented onthe SLM can be considered to be obtained by sampling andre-interpolating the ideal holographic image that is intended to bedisplayed on the SLM. Rate of sampling is determined by pixel pitch ofthe SLM, while the pixel aperture function of the SLM constitutes theinterpolating function. It is well known that when a signal is sampledin the space domain, its spectrum is periodically replicated in thespatial frequency domain. Thus, the spectrum of the ideal holographicimage that is intended to be displayed on the SLM is replicated as aresult of sampling, where these replicas are referred to as “higherdiffraction orders (HDO).” Since the pixel aperture function ofpractical SLMs are space limited functions (having Fourier transformsconsisting of decaying but non-limited tails), the replicas partiallysurvive in the Fourier transform of the final analog mask implemented bythe SLM, leading to observable higher diffraction orders.

As a simple example, FIG. 6 shows an SLM having a pixel pitch P at aneye relief distance D from a user's eye. The distance X betweendiffraction orders on the exit pupil plane can be approximated for smallangles as

$\begin{matrix}{X \approx \frac{D\; \lambda}{P}} & (1)\end{matrix}$

where λ is the wavelength of light incident on the SLM.

As discussed further below, various embodiments of the present inventionselect values for pixel pitch, expected eye relief distance, wavelength,and other parameters, such that the user's eye pupil forms an effectivespatial filter.

The hatch pattern shown in the SLM of FIG. 6 and other figures is not toany particular scale. The hatch pattern, when included, is used as avisual aid to differentiate SLMs from other optical components in thefigures, and is not meant to imply an actual pixel-pitch scale.

FIG. 7 shows a cross section of an SLM that generates noise beams andmultiple diffraction orders. SLM 710 may be either transmissive orreflective. FIG. 7 shows the light modulated by the SLM, but does notshow the illumination wave. The illumination wave may come from anydirection. The light wave distribution falling on the exit pupil plane220 includes the virtual-scene wave (the 0^(th) order), higherdiffraction orders (HDO), and noise beams.

The useful portion of the exit pupil plane is that portion that ideallyincludes the virtual-scene wave and nothing else. As shown in FIG. 7,noise beams and HDOs are not included in the useful portion of the exitpupil plane. As described further below, when a user's eye pupil issubstantially aligned with the useful portion of the exit pupil plane,the correct virtual scene represented by the virtual-scene wave isimaged on the user's retina.

FIG. 8 shows the cross section of FIG. 7 with a user's eye pupilfiltering out unwanted noise beams and diffraction orders. Embodimentsrepresented by FIG. 8 eliminate HDOs, noise beams, DC beams, conjugatebeams, and other possibly disturbing beams by using the eye pupil of theuser as a spatial filter. In these embodiments, no attempt to eliminatethe undesired beams is made (optically or computationally) within thenear-to-eye display device before these beams reach the exit pupilplane. However, the optical architecture of the system is designed andthe holographic image on the SLM is computed such that on the exit pupilplane, there is a useful portion within which only the virtual-scenewave exists, and all other undesired beams fall outside this region. Ifthe size of this useful portion is at least equal to the size of thepupil of the user, and (if needed) this useful portion is steered tofollow the pupil movements of the user, then the undesired beams areautomatically eliminated by the user's pupil and do not propagate to theretina. This technique, which we refer to as “pupil filtering,” has thebenefit of reducing the bulk within optical designs, but demands the SLMpixel pitch to be sufficiently small, or equivalently, spatial bandwidthof the SLM to be sufficiently high (see FIG. 6).

In some embodiments where pupil filtering is not applicable, opticalfilters (such as 4f filters) may be used within the system to eliminateHDOs and noise beams before they reach the exit pupil plane. However, inthese embodiments, the bulk within the system increases. Mostembodiments of the invention described herein employ pupil filtering,and therefore benefit from reduced bulk and weight.

FIGS. 9, 10, and 11 show multiple diffraction orders on an exit pupilplane having a useful portion. Each of FIGS. 9, 10, and 11 show multiplediffraction orders as black dots. The centermost diffraction order isthe virtual-scene wave, which includes the information desired to bepropagated to the retina. FIG. 9 also shows a schematic representationof the noise beams surrounding the virtual-scene wave. In operation, thenoise beams have a finite distribution not shown in the figures.

The ideal useful portion of the exit pupil plane includes all of thevirtual-scene wave and nothing else. Pupil filtering works when theuser's pupil is substantially aligned with the useful portion of theexit pupil plane such that the virtual-scene wave is allowed topropagate to the retina while everything else is filtered out. Inpractice, ideal pupil filtering may not always be achieved. For example,in some embodiments, the user's pupil substantially overlaps the usefulportion of the exit pupil plane (FIG. 10). These embodiments provideless-than-perfect pupil filtering.

Some embodiments generate a useful portion of the exit pupil plane largeenough so that it is at least the size of an expected pupil size.Physically, the minimum pupil width is typically assumed to be 2 mm.However, what is of concern to us is the physical size of the image ofthe pupil in front of the cornea (i.e., entrance pupil of the eye),which typically has a width slightly greater than 2 mm due to lensingeffect at the cornea. Three mm might be a typical minimum value. Hence,some embodiments of the present invention create a useful portion havinga width no smaller than about 3 mm. If the width of the useful portionis smaller than 3 mm, some part of the undesired beams may enter throughthe pupil, degrading the image quality at the retina. Further, someembodiments maintain the amount of average light power that is deliveredto the eye above a certain threshold, in order to guarantee that theuser's pupil size stays at the lower size limit when the display deviceis in use.

FIGS. 12-34 show various optical architectures suitable for use innear-to-eye display devices described herein. Some employ transmissiveSLMs and some employ reflective SLMs. Some block the real-world views tocreate a virtual reality, and some superimpose the real-world view onthe virtual scene to create an augmented reality. No near-to-eye displaydevice described herein is limited to any one (or any set) of theoptical architectures. In general, subsets of each of the opticalarchitectures may be considered as part of an illumination optics module(440, FIG. 4). Further, the optical architectures in many of the figuresbelow are shown for a single eye. In some embodiments, they arereplicated to create two sides of a display. Further, in someembodiments, when they are replicated, they are mirrored to providesymmetry.

FIG. 12 shows an optical architecture in which the SLM is placed on aconverging beam path, where the converging beam is obtained from a pointlight source 120 by an optical component with a positive refractivepower (here shown as a positive lens 1210) placed between the pointlight source 120 and the transmissive SLM 410. Note that in thisarchitecture, the point light source is actually imaged on the exitpupil plane 220. Therefore, the point light source is optically at aconjugate plane of the exit pupil plane. Also note that the SLM, underthe assumption that it is closer to the eye than the normal closestpoint of the human eye (25 cm), is not at a plane that is conjugate tothe retina. One advantage of this architecture is that the directivitypatterns of the light waves emerging from each pixel of the SLM are madeto almost completely overlap on the exit pupil plane. Thus, wherever theuseful portion is located, a uniform light power is intercepted fromeach pixel of the SLM. In this architecture, the SLM acts as the opticalmask that transforms the converging illumination beam to the part of thevirtual-scene wave that propagates to and fills the useful portion ofthe exit pupil plane. The spatial bandwidth requirement of the SLM isdirectly proportional to the width of the useful portion of the exitpupil plane. In order for the pupil-filtering technique to work, SLMbandwidth must be sufficiently large so that the useful portion isgreater than at least the expected minimum size of user's eye pupil. Thepixel pitch of the SLM must at least be smaller than the multiplicationof the wavelength of light produced by the point light source and thedistance between the SLM and the exit pupil plane divided by minimumsize of the eye pupil. When the SLM provides only certain type ofrestricted modulation, a smaller pixel pitch is needed, so that some ofthe additional SLM bandwidth can be used to distribute the noise beam.If the SLM provides real valued modulation (such as binary amplitude orphase modulation, or intensity modulation), the pixel pitch must behalved, since half of the bandwidth will be occupied by the conjugatebeam. In case the SLM generates an unmodulated DC beam, the usefulportion can be located at a slightly off axis eye position so that theDC beam can also be filtered by the eye pupil. Finally, the opticalcomponent that focuses the diverging light from the point light sourceto the exit pupil plane, in a practical implementation, might representa reflective element, such as an elliptical mirror, a spherical mirror,etc. Such a component both acts as a lens and also changes the opticalaxis.

FIG. 13 shows an optical architecture in which the SLM is illuminated bya diverging wavefront. The light modulated by the SLM, which has anoverall diverging character, is then collected by an eyepiece lens 1310and directed towards the eye. The point light source and the exit pupilplane are again conjugate planes. The SLM might or might not be at aplane that is conjugate to the retina depending on its position. In thisarchitecture, the eyepiece lens basically forms an image of the SLM,which might be virtual or real depending on the position of the SLM.This image of the SLM is referred to herein as the “effective SLM” andit appears to be illuminated by a converging wave. Thus, from theperspective of the effective SLM, the architecture is equivalent to thearchitecture shown in FIG. 12. Hence, the pupil-filtering techniqueworks if the pixel pitch of the effective SLM is sufficiently small asdiscussed in FIG. 12. In a practical architecture, a reflective surface,such as an elliptical, spherical, etc. mirror may be the opticalequivalent of the eyepiece lens illustrated here. This architectureconstitutes a convenient option for designing augmented-realitydisplays, especially in cases where the SLM is reflective andnon-transparent. In such cases, the SLM might be placed on the side ofthe eyeglass frame, and the light from the SLM can be directed towardthe eye by a semitransparent reflective surface, which is the opticalequivalent of the eyepiece lens illustrated here. Such architectures areillustrated in subsequent figures.

FIG. 14 shows an optical architecture with a point light source and SLM,with no other components with refractive power. In contrast to theprevious two cases, the point light source is not at an opticalconjugate plane of the exit pupil plane, since it is not imaged on theexit pupil plane. Similarly, the SLM is not at an optical conjugateplane of the retina. The greatest advantage of this architecture is itssimplicity, thus the potential for realizing near-to-eye display deviceswith quite thin form factors, since no components other than the SLM andpoint light source are present. However, since the SLM is illuminatedwith diverging light, and the light from the SLM retains its overalldiverging character at the exit pupil plane, directivity patterns oflight waves from each pixel of the SLM do not overlap on the exit pupilplane. Hence, there is a variation in the power level that isintercepted from each pixel of the SLM, leading to a similar variationon the virtual scene. This variation can partially be reduced duringcomputation of the holographic image to be displayed on the SLM.However, some variation and dark regions will inevitably be present.

Some embodiments use SLMs with lower fill factors. In these embodiments,though there is a loss in light efficiency, the directivity patterns ofSLM pixels become uniform, i.e., SLM pixels optically behave closer toisotropic point light sources, and the intensity variation describedabove no longer exists. Further, in embodiments where the SLM generatesan unmodulated DC beam, that beam is not focused to a single spot on theexit pupil plane, but spreads over a large area. Hence, some part of itenters into the useful portion. However, since the energy is spread out,only a little portion of the unmodulated DC beam is intercepted, and therelated background noise on the retina is quite low if not perceivableat all.

FIG. 15 shows an architecture in which an SLM is illuminated in a timesequential manner by an array of point light sources. As an example,five point-light sources PS1 to PS5 are illustrated where PS3 is assumedto be on. When only one of the point light sources is considered, thearchitecture is the same with the architecture in FIG. 14, and thenon-uniform brightness problem discussed in FIG. 14 is present. However,as the point light source that is switched on changes, the part of theSLM that contributes to the useful portion with the highest powerchanges. Alternatively, the power contributed by a particular section ofthe SLM to the useful portion changes as the point light source that isturned on changes. In particular, the number and positions of pointlight sources are arranged such that when time averaged, every part ofthe SLM sends equal power to the useful portion. Therefore, the pointlight source array enables us to obtain a uniform variation ofbrightness in the field of view by time integration of retinal imagescreated by different point light sources. Embodiments represented byFIG. 15 demand a higher frame rate SLM than previously describedembodiments. The higher frame rate is driven in synchronism with thepoint light sources, and the deployment of multiple point light sources.Also, for each point light source, the holographic image on the SLM mustbe updated according to the new position of the illumination wave.Therefore, multiple holographic images need to be computed for eachvideo frame of the virtual scene.

In general, point light sources should be turned on one at a time onlyif all light sources significantly illuminate every part of the SLM andno crosstalk at all among reconstructions by different point lightsources is tolerable. In some embodiments capable of tolerating someweak level of crosstalk, it is possible to group the light sources andturn each group on at a time. For example, point light sources PS1, PS3,and PS5 may form the first group, and PS2 and PS4 may form the secondgroup. The crosstalk between the point light sources in any of thesegroups is weak due to the fact there is sufficient separation betweenthe light sources and the light power received from a part of the SLM isdominated by one of the sources. In this way, the demand on SLM framerate is decreased. Note that in this strategy, the holographic image oneach region of the SLM is computed according to the point light sourcethrough which the highest power is delivered to the useful portion fromthat region.

FIG. 16 shows an architecture, which is similar to the architectureillustrated in FIG. 15 with the difference that all point light sourcesare simultaneously turned on, and the directivity angles of point lightsources are constrained, possibly by apertures placed in front of thepoint light sources. In this architecture, the SLM surface is dividedinto a number of nonoverlapping regions (labeled 1-5 for examplepurposes), where each of these regions are essentially illuminated byonly one of the point sources. Therefore, the light wave in the usefulportion is formed by the superposition of the waves from multiple pointlight sources. The holographic image on each of the regions of the SLMis computed according to the corresponding light source, and the finalholographic image displayed on the SLM is obtained by concatenatingthese individual holographic images. One advantage of this architectureover the architecture shown in FIG. 15 is that there is no need for ahigher frame rate SLM, and the computation of a single holographic imagefor each video frame is sufficient. One drawback, however, is that theapertures placed in front of the point light sources somewhat increasethe bulk of the system. In addition, some diffraction artifacts andcorresponding resolution loss will be observed for virtual-scene pointsthat lie close or in the direction of the boundary regions of the SLMthat are illuminated by different point light sources.

Some embodiments use a second group of point light sources interspersedwith the existing group, such that the second group again divides theSLM surface into non-overlapping regions, but this time with boundariesfalling into the middle of the regions formed by the first group oflight sources. In these embodiments, the first and second groups oflight sources are turned on in a time sequential manner. Object pointsthat lie close to the one set of boundaries might be skipped when thecorresponding group of light sources are turned on, and they may bedisplayed only when the other group of light sources are turned on, withdoubled intensity so that average power stays the same. In this way,diffraction artifacts and resolution loss for virtual-scene points thatlie close to the boundary regions can be avoided, however, twice a framerate is demanded from the SLM.

FIG. 17 shows an optical architecture in which a reflective SLM isplaced directly in front of the user's eye. In FIG. 17, a reflective SLM110 is placed directly in front of the eye and is illuminated by a pointlight source 120 mounted on the side of the eyeglass. The system isoptically equivalent to the system depicted in FIG. 14, and constitutesa non-see-through display since the SLM blocks the vision of real world.

FIG. 18 shows an architecture in which the SLM is placed such thatreal-world vision is not blocked. In FIG. 18, a reflective SLM 110 isplaced at a position such that real-world vision is not blocked. The SLMis illuminated by a point light source 120 mounted on the side of theeyeglass. The light reflected from the SLM 110 is directed to the user'seye by a beamsplitter 1810. The system is optically equivalent to thesystem depicted in FIG. 14, and constitutes a see-through display.

In FIG. 19, a transmissive SLM 410 is placed directly in front of theeye such that real-world vision is not blocked, however, as thereal-world light passes through the SLM, the image of the real worldmight be slightly corrupted. The SLM is illuminated by a point lightsource 120 mounted on the side of the eyeglass at a location that isfurther to the eye than the SLM. The system is optically equivalent tothe system depicted in FIG. 14, and constitutes a see-through displaywith some degradation of real-world view.

In FIG. 20, a transmissive SLM 410 is placed at a position so thatreal-world vision is not affected by its presence. The SLM isilluminated by a point light source 120 mounted on the side of theeyeglass. The light transmitted by the SLM is directed to the eye by abeamsplitter 1810. The system is optically equivalent to the systemdepicted in FIG. 14, and constitutes a see-through display with nodegradation of real-world view.

FIG. 21 shows an optical architecture in which a reflective SLM 110 isplaced in front of the user's eye. In FIG. 21, a look at display isimplemented with a reflective SLM. A positive lens 2110 is placed infront of the SLM. The focal length of the positive lens is equal to eyerelief distance. The lens converts the diverging wave from the pointlight source 120 to a collimated beam, which hits the SLM with a slightangle, gets modulated and reflected, and passes once again through thesame lens which now acts as an eyepiece lens and directs the lighttowards the pupil. The system is optically equivalent to the system inFIG. 13.

FIGS. 22-28 show optical architectures in which real-world vision is notblocked by the SLM. In FIG. 22, the reflective SLM 110 is placed to theside of the eyeglass frame so that the reflective SLM does not block thereal-world view. An additional beamsplitter 1810 is used to direct SLMlight to the eye pupil of the user. The system is optically equivalentto the system in FIG. 12, and constitutes a see-through display.

In FIG. 23, a see-through display is implemented with a transmissive SLM410. The diverging light wave from a point light source 120 is convertedto a converging wave by a positive lens 1210. The converging wave passesthrough the SLM and gets modulated. The SLM wave is directed towards theeye with a beamsplitter 1810. Though the SLM is transmissive, the lensand the SLM are both placed before the beamsplitter so that real-worldview is not affected by their presence. The system is opticallyequivalent to the system in FIG. 12.

In FIG. 24, a see-through display is implemented with a transmissiveSLM. Essentially, the places of the lens and the SLM in FIG. 23 areinterchanged. The system is optically equivalent to the system in FIG.13.

In FIG. 25, a see-through display with a reflective SLM 110 isillustrated. The system is optically equivalent to the system in FIG.13, where the eyepiece lens is replaced by the semi-transparentreflector 2510 placed in front of the eye. The reflector 2510 can eitherbe a single piece curved component, such as an elliptical or sphericalmirror, or it can be a flat component with an array of micromirrors withdifferent tilt angles.

In FIG. 26, a see-through display with a reflective SLM is illustrated.The system is optically equivalent to the system in FIG. 13. Thebeamsplitter on the right and the lens form a virtual image of the pointlight source, and SLM is illuminated by a diverging spherical wave whichseems to emerge from the said virtual image of the point light source.This wave gets modulated, and then is bent towards the eye pupil withthe combination of lens and curved mirror. The architecture isadvantageous in that it is compact and provides undistorted see-throughvision.

In FIG. 27, a see-through display with a transmissive SLM 410 isillustrated. The system is optically equivalent to the system in FIG. 13and different from the system in FIG. 25 only in that the SLM istransmissive.

In FIG. 28, a see-through display with a transmissive SLM 410 isillustrated. The system is optically equivalent to the system in FIG. 13and different from the system in FIG. 27 only in that beamsplitter 1810is included.

FIG. 29 shows an optical architecture in which an SLM and reflector arecombined. As shown in FIG. 29, the SLM is fabricated directly on thesemitransparent reflector. The diverging light from the point lightsource 120 illuminates the SLM, which is directly fabricated on top of asemi-transparent reflector. The SLM-reflector combination can beconsidered as a single device, which is similar to LCoS SLMs, butfabricated on a transparent substrate. Because the SLM and reflector areessentially a single device, any light ray hitting the SLM also exitsthe SLM at the same point. The system is optically equivalent to FIG.13.

FIG. 30 shows a reflector based solution for modulation in a singledirection. FIG. 30 illustrates an embodiment of the invention in which atransmissive SLM 410 is placed between a semi-transparent reflector 2510and the eye to constitute a see-through display. In some embodiments,the reflector and the SLM are separate devices, with considerable spacein between. If the polarizers 3020 and 3010 were not present, the waveemanated from the point light source 120 would get modulated by thetransmissive SLM twice: firstly during the initial passage, secondlyafter getting reflected from the semi-transparent reflector. This doublemodulation is undesired especially when some of the incident light raysare modulated by different sections of the SLM. In order to eliminatethis double modulation, light wave emitted by the point light source isfirst passed through a polarizer 2920. As the transmissive SLM, a liquidcrystal SLM in Parallel Aligned Nematic (PAN) mode may be used, wherethe LC director axis of the liquid crystal is orthogonal to the axis ofthe polarizer 3020 that is placed in front of the point light source.Then, the light emanated by the point source does not get modulated bythe SLM during the first passage. After passing the SLM, the light wavepasses through a 45-degree polarization rotator 3010, then getsreflected from the semi-transparent reflector 2510, and then passes onceagain through the 45-degree polarization rotator 3010 after which itspolarization becomes parallel to the LC director of the SLM. Then thewave enters the SLM once again, and gets modulated this time. In thismanner, double modulation is avoided and the incident light wave ismodulated by the SLM only during its second passage.

FIG. 31 shows a virtual-reality (VR) architecture with SLM tiling. Lightfrom point light source 120 is collimated by collimation lens 3150,passed through a polarizer 3130, and split into two with beamsplitter3114. One portion is fed to the first reflective SLM 3110, and the otherportion is fed to the second reflective SLM 3120. Modulated light comingfrom the SLMs are joined by beamsplitter 3112 and then passed through acommon eyepiece lens 3140 and directed to the eye. The architecture isparticularly useful when it is not possible to place SLMs side by sidedue to their external frames that contain the electronic controlcircuitry. The SLMs used in the architecture can be identical.

FIG. 32 shows a VR architecture with SLM tiling. This architecture has asmaller form factor than the architecture in FIG. 31, but it requiresthe LC director axis of the SLMs to be perpendicular to each other. Inaddition, the R1=T2R2 condition is required so that both SLMs receiveequal amount of light power.

FIG. 33 shows a VR display architecture. A concave mirror 3310, such asused in telescopes, has an opening. The diverging waves emitted by twopoint-light sources 120 are converted to two pieces of converging wavesby the mirror. The converging waves illuminate the reflective SLM 110.The light modulated by the SLM propagates to the exit pupil planethrough the opening between the mirrors.

FIG. 34 shows two tiled SLMs to double resolution. Two identicalreflective SLMs 110A and 110B are placed facing opposite surfaces ofbeamsplitter 3420. The SLMs are illuminated by collimated light sentfrom an illumination optics module 3430. The light emerging at 3450 isequivalent to the light generated by a single SLM that is obtained byadding the complex transmittances of the two SLMs. The SLMs arepositioned such that they are offset on the transverse plane by half apixel pitch with respect to each other during the addition. Theequivalent SLM 3410 then has a pixel pitch that is half the pixel pitchof each reflective SLM. The pixel aperture function of the equivalentSLM is the same as the pixel aperture function of the reflective SLM.Since the effective SLM has a higher pixel pitch, its bandwidth and theangular separation between diffraction orders are increased. Such astructure can be used to enlarge the size of the useful portion that canbe obtained.

FIG. 35 shows a near-to-eye display device with a high-resolutionportion and a low-resolution portion. The high-resolution portion isprovided by inset 3510 and low-resolution portion is provided byperipheral imaging device 3520. In some embodiments, peripheral imagingdevice 3520 includes a microdisplay device such as an organic lightemitting diode (OLED) display, a liquid crystal display (LCD), or areflective LCD.

In some embodiments, the high-resolution inset is an SLM that provides avirtual scene to the user as described above. In these embodiments, theSLM has a fixed location and so does the high-resolution inset withinthe resulting display. In these embodiments, near-to-eye display device3500 includes an SLM that provides about 30-40 degrees high-resolutioncentral foveal vision with natural depth cues, and a regular 2D displaythat provides a low-resolution peripheral image. The idea presented heredepends on the promise that though the human eyes have a largeFOV—around 170 degrees—a very small portion of this FOV (around 6degrees) constitutes sharp foveal vision at a time. Humans typicallyenlarge the FOV for sharp foveal vision to about 30-40 degrees by eyemotion before resorting to head motion. Therefore, a display thatsupports a high-quality foveal vision within a FOV of 30-40 degrees, andsupplements this with a low-quality peripheral vision will be aneconomical solution for large FOV designs. The image provided by the SLMcarries all natural depth cues in addition to being high resolution. Theeye can focus on the virtual objects seen through the SLM as in naturalvision. The peripheral image provided by the regular 2D display is notfocused on the retina and is low resolution. However, it stillestablishes a degree of peripheral awareness.

FIG. 36 shows a high-resolution image for foveal vision and lowerresolution for peripheral vision. Image 3600 represents an image seen bya user using near-to-eye display device 3500. The part of the virtualscene that falls in the central-vision part of the FOV appears as ahigh-resolution image, while the part that falls in theperipheral-vision part appears as a low resolution and defocused image.

FIG. 37 shows the high-resolution image being steered to a user's pupilposition. Some embodiments provide for the high-resolution image to bemoved within the field of view. Examples of these embodiments aredescribed with reference to figures that follow. Image 3700 representsan image seen by a user when the user's pupil is tracked as the userlooks to the right within the FOV. The high-resolution inset is steeredto follow the user's eye movement.

FIGS. 38 and 39 show a display system with a rotating hologram module tocreate a steerable high-resolution image. In some embodimentsrepresented by FIGS. 38 and 39, only the 6-10 degree portion of the FOVis provided by the SLM at a single time. In other embodiments, more than6-10 degrees is provided at a time. The rest of the FOV is covered by aregular 2D display. Pupil movements of the user are tracked, and thehologram module 3810 is rotated based on those movements to steer theSLM light towards the pupil. Part of the 2D display image that lieswithin the central-vision region is temporarily blackened, so that thecentral vision is formed only by the SLM and thus is high resolution.The reflector is designed such that the SLM light is directed towardsthe eye pupil for any position of the eye pupil.

Rotating hologram module 3810 is shown with an SLM, lens, beamsplitter,and light source. Any of the optical architectures described herein maybe employed within rotating hologram module 3810 without departing fromthe scope of the present invention.

In some embodiments, LCD 3820 is used as peripheral imaging device 3520(FIG. 35), and rotating hologram module 3810 illuminates a portion ofLCD 3820 to create the high-resolution inset 3510 (FIG. 35). Rotatinghologram module 3810 may be physically location on the frame ofnear-to-eye display device 35. For example, rotating hologram module3810 may be co-located with a point light source 120.

FIG. 39 shows pupil tracker 3910 tracking movement of the user's eye 210and actuator 3920 used to rotate rotating hologram module 3810. When theuser moves eye 210, pupil tracker 3910 sends a signal to actuator 3920to cause the hologram module to rotate. Pupil tracker 3910 may includeany suitable components capable of performing as described. For example,pupil tracker 3910 may include one or more cameras, one more lightsources (e.g., infrared), and a processing element to interpret thepupil-tracking data and to command actuator 3920. Actuator 3920 mayinclude any type of component capable of performing as described. Forexample, actuator 3920 may be a stepper motor or series of steppermotors coupled to rotating hologram module 3810.

FIG. 40 shows a portion of a near-to-eye display device having a movingplatform. Moving platform 4010 moves within the field of view of theuser. Moving platform 4010 is actuated by circuits (not shown) mountedon the near-to-eye display device, or connected to the near-to-eyedisplay device with cabling or wirelessly. In some embodiments, movingplatform includes light sources and/or SLMs. In these embodiments, thelight sources and/or SLMs are driven by circuits (not shown) mounted onthe near-to-eye display device, or connected to the near-to-eye displaydevice with cabling or wirelessly. Various embodiments of movingplatforms are now described.

FIG. 41 shows a moving platform upon which an SLM bar that covers about30-40 degrees of central FOV is mounted, along with two LED bars each ofwhich covers about 30 degrees of peripheral FOV. The SLM bar includes aplurality of pixels, the spacing of which satisfies the criteriadescribed herein with respect to the useful portion of the exit pupilplane. The LED bars may include any number of pixels. In someembodiments, the resolution of the LED bars is less than the resolutionof the SLM bar. The entire platform 4010 can move up and downperiodically to scan the vertical direction. The display is consideredsee-through since the moving platform does not continuously block anypart of the user's FOV, but does so only for a short duration of time.Both the SLM bar and the LED bar have high refresh rates.

FIG. 42 shows a portion of a near-to-eye display device having a movingplatform with an array of bars. Moving platform 4210 includes more thanone bar that moves up and down in the vertical direction to fill theFOV. Moving platforms that include a plurality of bars, such as platform4210 are also referred to herein as “slotted platforms.” Moving platform4210 is actuated synchronously with the SLM data being driven on thevarious SLM elements on moving bar 4210. The idea is similar to FIG. 40,except for the fact that an array of bars are used so that each of thebars needs to scan a smaller vertical range, relieving the frame-rateconstraint on the SLM bar and LED bar.

Portions of moving platform 4210 are considered to include amicrodisplay. For example, the portions of bars 4010 that include LEDsand the LED bars above and below bars 4010 constitute a microdisplay. Insome embodiments, microdisplays on moving bars have a lower resolutionthan SLM bars. Also in some embodiments, microdisplays on moving barshave a greater pixel pitch than SLM bars.

FIG. 43 shows a moving platform having an array of bars upon which SLMsand microdisplays are mounted. Moving platform 4210 includes a pluralityof bars equivalent to 4010, and a plurality of bars that only includeLEDs. The SLM bar is mounted only on the bars in the middle so that30-40 degrees of FOV is covered in the vertical direction as well. Thetop and bottom bars only consist of LEDs, since they are not responsiblefor central foveal vision but only peripheral vision.

FIG. 44 shows an even simpler design where only a small SLM is mountedon the middle bar for central vision, while two more LED bars are placedto provide peripheral vision. At a single time, the SLM bar only coversabout 6-7 degrees of horizontal FOV. For a fixed position of eye pupil,the bar only scans in the vertical direction to cover 6-7 degrees ofvertical FOV as well. When the eye pupil moves, the SLM bar also movesin the horizontal direction to cover the portion of the FOV for centralvision. In some embodiments, all bars shown move as described, and inother embodiments, only the middle bar with the SLM moves as described.

FIG. 45 shows a near-to-eye display device with a transducer to interactwith a user for calibration. Near-to-eye display device 4500 is similarto near-to-eye display device 100 (FIG. 1) with the addition ofadjustment knob 4510. Adjustment knob 4510 is an example of a transducerthat allows the user to interact with the near-to-eye display device.For example, in some embodiments, near-to-eye display device 4510 mayperform calibration actions in which the user is asked to providefeedback using the transducer. Various calibration embodiments are nowdescribed.

FIGS. 46, 48, 50, and 52 show flowcharts of calibration methods inaccordance with various embodiments of the present invention. In someembodiments, these methods, or portions thereof, are performed by anear-to-eye display device, embodiments of which are shown in, anddescribed with reference to, the figures of this disclosure. In otherembodiments, these methods are performed by a computer or an electronicsystem. The various calibration methods are not limited by theparticular type of apparatus performing the method. Further thedisclosed actions in the calibration methods may be performed in theorder presented, or may be performed in a different order. Also, in someembodiments, some actions listed in the figures are omitted whileperforming method embodiments.

In calibration embodiments according to FIG. 46, a user is prompted toidentify a type of any visual disorder. An example image to prompt auser is shown at 4710 (FIG. 47). Once the user has entered a type ofdisorder, the near-to-eye display device may display a chart from whichthe user may make a selection. For example, in the example execution ofthe method shown in FIG. 47, the user has selected myopia, and thesystem presents a chart prompting the user to select the smallest letterthe user can comfortably read. In some embodiments, a user may make aselection by looking at it, in which case built in pupil-trackinghardware can interpret the selection. In other embodiments, a user mayinteract with a transducer, such as adjustment knob 4510 (FIG. 45) tomake the selection, and in still further embodiments, a user mayinteract with a touch-sensitive portion of the display area on thenear-to-eye display device.

At 4620, a light wave distribution is modified to present the user withat least one test image intended to determine a degree of the visualdisorder suffered by the user. For example, in some embodiments, asingle image such as that shown at 4730 (FIG. 47) is shown to the user.In other embodiments, multiple images such as those shown at 4920 (FIG.9) are shown to the user.

At 4630, feedback is received from the user regarding the at least onetest image. In some embodiments, this corresponds to a user selecting animage using a transducer. In other embodiments, this corresponds to auser turning an adjustment knob. For example, as a user interacts withadjustment knob, the image at 4730 may be focused at different distancesuntil the user's myopia has been overcome.

At 4640, the light wave distribution is modified to correct for thevisual disorder suffered by the user. This is shown at 4740 (FIG. 47).The different images displayed are generated using an SLM as describedabove. Visual disorders may be corrected using the computation of theSLM data. Computation of SLM data is described further below.

In calibration embodiments according to FIG. 48, a light wavedistribution is modified to present the user with a plurality of testimages intended to determine a type of visual disorder (if any) sufferedby the user. For example, in some embodiments, images such as that shownat 4910 (FIG. 49) are shown to the user. At 4820, feedback is receivedfrom the user regarding the plurality of test images. In someembodiments, this corresponds to a user selecting an image using atransducer. In other embodiments, this corresponds to a user turning anadjustment knob. In still further embodiments, this corresponds to auser interacting with a touch-sensitive portion of the display.

At 4830, the type of visual disorder suffered by the user is determinedbased on the feedback received. In the example execution of the methodshown in FIG. 49, the user has selected an image corresponding tomyopia.

At 4840, the light wave distribution is modified to present the userwith a second plurality of test images intended to determine a degree ofthe visual disorder suffered by the user. For example, in someembodiments, multiple images such as those shown at 4920 (FIG. 9) areshown to the user.

At 4850, additional feedback is received from the user regarding thesecond plurality of test images. In some embodiments, this correspondsto a user selecting an image using a transducer. In other embodiments,this corresponds to a user turning an adjustment knob or interactingwith a touch-sensitive display. In some embodiments, 4840 and 4850 areperformed more than once to determine the proper correction to beapplied to correct the user's visual disorder.

At 4860, the light wave distribution is modified to correct for thevisual disorder suffered by the user. This is shown at 4930 (FIG. 49).The different images displayed are generated using an SLM as describedabove. Visual disorders may be corrected using the computation of theSLM data. Computation of SLM data is described further below.

In calibration embodiments according to FIG. 50, the user is prompted toenter the type and degree of the visual disorder at 5010. Example imagesto prompt a user are shown at 5110 and 5120 (FIG. 51). Once the user hasentered the type and degree of disorder, the near-to-eye display devicemodifies a light wave distribution to present the user with at least onetest image intended to correct for the visual disorder suffered by theuser at 5020. This is shown at 5130.

At 5030, feedback is received from the user regarding the at least onetest image. In some embodiments, this corresponds to a user selecting animage using a transducer. In other embodiments, this corresponds to auser turning an adjustment knob or interacting with a touch-sensitivedisplay. For example, as a user interacts with adjustment knob, theimage at 5130 may be focused at different distances until the user'smyopia has been overcome

At 5040, the light wave distribution is modified to correct for thevisual disorder suffered by the user. This is shown at 5140 (FIG. 51).The different images displayed are generated using an SLM as describedabove. Visual disorders may be corrected using the computation of theSLM data. Computation of SLM data is described further below.

In some embodiments, user profiles are stored within the near-to-eyedisplay device for later retrieval. Also in some embodiments, thecalibration methods described also provide actions to allow forbrightness, contrast, and color correction. Any type of visual settingmay be applied and any type image enhancement may be incorporatedwithout departing from the scope of the present invention.

FIG. 52 shows a flowchart of calibration methods in accordance withvarious embodiments of the invention. Methods represented by FIG. 52differ from the previously described calibration methods in that methodsrepresented by FIG. 52 interact with one or more actuators on thenear-to-eye display device to correct for anomalies.

At 5210, at least one test image is displayed. This is shown in FIG. 53.Note that in all calibration embodiments, test images are not actuallydisplayed on an eyeglass lens, but rather, test images are made to bepart of a virtual scene using the SLM and pupil-filtering methodsdescribed above.

In some embodiments, the user is shown multiple test images that are atdifferent depths and transverse positions. During this procedure, twocameras (that are mounted on the HWD and well calibrated with respect toeach other) may record the position of her pupils at 5220. As shown inFIG. 53, the cameras may be infrared cameras. Example images that may berecorded are shown in FIG. 54.

At 5230, the recorded images are analyzed for user characteristics suchas interpupil distance (IPD), and convergence when viewing 3D images.For example, images of the eyes captured by the cameras may be analyzedto determine the convergence points of the eyes. In some embodiments,the images from the infrared cameras are processed to preciselyunderstand the eye relief distance, orientation of the display withrespect to the head and eyes of the user, inter pupillary distance ofthe user, etc. In particular, the sizes of the iris, limbus, andlocations of and distances between Purkinje images (images of infraredlight sources generated by reflections from different layers of user'seye) are detected and analyzed to get the best prediction about thevalues of the relative positional parameters between the eyeglass andthe user's eyes.

Based on this information, the near-to-eye display device may cause theactuators 5310, 5320, and 5330 to change physical characteristics of thedevice to accommodate a user's physical characteristics. For example,the near-to-eye display device may adjust SLM position, position oflight sources, distance between the two SLMs, and the like. Examples areshown in FIG. 55.

The various embodiments of the present invention provide for precisecalibration between left and right display units, which enables correctand comfortable 3D vision. Once the display is calibrated for a user,she can save the parameters in a user profile stored in the system. Whenshe wears the display the next time—after another user—she can selecther own profile. In this way, the display can be shared by multipleusers in a convenient manner.

Further, when a user wears the display, the cameras may take a quickphoto of the eyes, and use iris-recognition algorithms to recognize auser who used the display before. Then, automatic re-calibration of thedisplay for that user can be performed.

FIG. 56 shows a flowchart representing computation of SLM data. In someembodiments, method 5600, or portions thereof, is performed by anear-to-eye display device, embodiments of which are shown in previousfigures. In other embodiments, method 5600 is performed by a computer oran electronic system. Method 5600 is not limited by the particular typeof apparatus performing the method. The various actions in method 5600may be performed in the order presented, or may be performed in adifferent order. Further, in some embodiments, some actions listed inFIG. 56 are omitted from method 5600.

As described previously, in various embodiments of the presentinvention, the display system delivers, to the useful portion, the lightwave that would be generated by a virtual scene. In particular, an SLMis the component through which the information about the virtual sceneis imposed on the light waves generated by the light source or sourceswithin the display. Therefore, the SLM transforms thenon-information-bearing wave illuminating it to the information-bearingwave part of which is delivered to the useful portion. Under this view,the main computational steps to determine SLM data for a frame of video(for a generic architecture) are as follows:

1. Compute the “2D complex-valued profile of virtual-scene wave onuseful portion” as shown at 5610.

2. Back-propagate the “2D complex-valued profile of virtual-scene waveon useful portion” computed in step 1 to the SLM plane—possibly passbackwards through intermediate components—find the “ideal 2Dcomplex-valued profile of wave at the exit of SLM” as shown at 5620.

3. Find the “2D complex-valued profile of wave illuminating the SLM”, bypropagating the wave emitted by the point light source and tracing itthrough the possibly present intermediate components as shown at 5630.

4. Determine the “2D ideal analog complex-valued SLM transmittance”,which is the transmittance the SLM must implement as the multiplicativemask profile that transforms the “2D complex-valued profile of waveilluminating the SLM” computed in step 3 to the “ideal 2D complex-valuedprofile of wave at the exit of SLM” computed in step 2 as shown at 5640.

5. Apply appropriate prefiltering to the “2D ideal analog complex-valuedSLM transmittance” to compensate for the effects of SLM structure(sampling and interpolation) to arrive at the “2D ideal complex-valueddiscrete SLM image” that needs to be displayed on the SLM as shown at5650.

6. Quantize and encode the ideal “2D ideal complex-valued discrete SLMimage” obtained in Step 5 into a phase-only, binary, amplitude-only,etc. representation depending on the modulation capabilities of the SLM,and find the “2D actual digital SLM image” that is suitable fordisplaying on the SLM as shown at 5660. During this step, actions aretaken to make sure that the resulting noise beams fall outside theuseful portion of the exit pupil plane.

A more detailed explanation of computation steps follows.

1. Computation of “2D complex-valued profile of virtual-scene wave onuseful portion.”

In various embodiments of the present invention, the user sees a virtualscene by intercepting the light wave delivered to the useful portion ofthe exit pupil plane. Therefore, the computational procedure starts bythe computation of the light wave that should be delivered to the usefulportion, which is referred to herein as the “2D complex-valued profileof virtual-scene wave on useful portion”. This wave carries all theinformation about the virtual scene. Ideally, the display systemdelivers only this wave within the useful portion and nothing else.However, as discussed above, due to the restrictions of the SLMs,several undesired beams are also generated by the SLM and they propagateto the exit pupil plane, but hopefully fall outside the useful portion.In order to compute the “2D complex-valued profile of virtual-scene waveon useful portion”, first of all, we need a computer-graphics model torepresent the virtual scene. Various models exist in the literature torepresent virtual scenes on a computer, ranging from simple point cloudmodels to more complicated polygon mesh based models.

In some embodiments of the presented invention, a point cloud model isused to represent a virtual scene. In this model, the virtual scene isassumed to consist of a finite number of point light sources. Dependingon the location of the useful portion, some of these virtual-scenepoints are visible, while some others are non-visible, due to occlusion.The “2D complex-valued profile of virtual-scene wave on useful portion”can be computed (for a single wavelength) by superposing the divergingspherical waves emitted by each of the visible virtual scenes on theuseful portion as follows:

$\begin{matrix}{{{U_{vs}\left( {x,y} \right)} = {\sum\limits_{i = 1}^{M}\left\lbrack {\sqrt{I_{i}}\left( e^{J\; \varphi_{i}} \right)\frac{e^{({J\frac{2\pi}{\lambda}R_{i}})}}{R_{i}}} \right\rbrack}}{{where}\text{:}}} & (2) \\{{R_{i} = \sqrt{\left( {x_{i} - x_{i}} \right)^{2} + \left( {y - y_{i}} \right)^{2} + z_{i}^{2}}},} & (3)\end{matrix}$

U_(vs)(x, y) is the 2D complex-valued profile of virtual-scene wave onuseful portion,

M is number of virtual-scene points that are visible,

I_(i) is the intensity of the i^(th) virtual-scene point,

φ_(i) is the phase of the i^(th) virtual-scene point,

(x_(i), y_(i), z_(i)) is the location of the i^(th) virtual-scene point,

λ is the wavelength, and

J=√{square root over (−1)}.

The computation according to the formula above applies for a singlecolor. Therefore, the computation should be repeated for all differentcolor components in a multicolor display embodiment.

The computation according to the formula above also applies only for acertain position of the pupil position. In some embodiments of thepresent invention, providing a motion parallax upon changing positionsof eye pupils of the user (as a result of eye or head motion) is notcritical or demanded at all. In these cases, the computation in theabove equation does not need to be updated, and it is sufficient to onlysteer the display light to the new position of the eye pupil, usingmechanisms that are described more fully below.

In some other embodiments of the present invention, it is desired forthe system to provide a motion parallax. In that case, when the positionof the eye pupil changes, which corresponds to a change in perspectivefrom which the virtual scene is viewed, one needs to identify the newset of virtual-scene points that become visible, and the new set ofoccluded virtual-scene points. Then, the “2D complex-valued profile ofvirtual-scene wave on useful portion” should be recalculated asdescribed above.

In some embodiments of the present invention, the polygon mesh modelsmight be more suitable, especially when system compatibility with orexpandability on existing computer-graphics hardware and software is ofconcern. In that case, most of the computer-graphics-related tasks suchas occlusion, shading, etc. can be accomplished on a graphicalprocessing unit (GPU). In particular, for a viewpoint of interest (or,for a given position of the useful portion), the virtual scene, which isrepresented by polygon mesh models, can be rendered by a GPU, whichhandles any occlusion, shading etc. effects as currently done withstandard GPUs embedded in computers, smart phones etc. Then, the finalrendered RGB image, as well as the depth or z buffer data (which is astandard piece of data stored in GPUs and represents the distance ofeach rendered virtual-scene point to viewpoint under consideration) canbe imported from the GPU. For each pixel of the RGB image, a sphericalwave term, emitted from the depth location of the point as inferred fromthe z or depth buffer, can be superposed to compute the “2Dcomplex-valued profile of virtual-scene wave on useful portion”, asdescribed above.

2. Computation of the “ideal 2D complex-valued profile of wave at theexit of SLM”

This step involves numerically back-propagating the “2D complex-valuedprofile of virtual-scene wave on useful portion” through any opticalcomponents that lie in the pathway between the exit pupil plane and theSLM, including sections of free space, and determining the “ideal 2Dcomplex-valued profile of wave at the exit of SLM”. Here, the identifier“ideal” stresses that if this field were present at the exit of the SLM,the light wave distribution on the exit pupil plane would not consist ofany undesired components.

In some embodiments of the present invention, the SLM is placed directlyin front of the user's eye. Therefore, there are no optical componentsin between except for a section of free space. In this case, the “ideal2D complex-valued profile of wave at the exit of SLM” can be found as:

$\begin{matrix}{{{U_{ExS}\left( {x,y} \right)} = {\left\{ {{U_{vs}\left( {x,y} \right)}{W_{up}\left( {x,y} \right)}} \right\}**{h_{- D_{er}}\left( {x,y} \right)}}}{{where}\text{:}}} & (4) \\{{W_{up}\left( {x,y} \right)} = \left\{ {\begin{matrix}1 & {{within}\mspace{14mu} {useful}\mspace{14mu} {portion}} \\0 & {{outside}\mspace{14mu} {useful}\mspace{14mu} {portion}}\end{matrix},} \right.} & (5) \\{{{h_{D}\left( {x,y} \right)} = {\frac{D}{J\; \lambda}\frac{e^{J\frac{2\pi}{\lambda}R}}{R^{2}}}},} & (6) \\{{R = \sqrt{x^{2} + y^{2} + D^{2}}},} & (7)\end{matrix}$

U_(ExS)(x, y) is the ideal 2D complex-valued profile of wave at the exitof SLM,

U_(vs)(x, y) is the 2D complex-valued profile of virtual-scene wave onuseful portion,

W_(up)(x, y) is the aperture function of useful portion, and

h_(D)(x, y) is the impulse response of free-space propagation.

In some embodiments of the present invention, there are some otheroptical components between the SLM and exit pupil plane. In that case,in addition to the free-space-propagation computations between differentcomponents, detailed wave optics models to account for the behavior ofthe components should be developed. These models should relate the lightwave distribution at the entrance side to the light wave distribution atthe exit side of a component. Note that when the model for eachcomponent is sufficiently detailed; aberrations, distortions,diffraction effects, and other effects introduced by the componentsbetween the eye and the SLM are incorporated into the computationsautomatically.

3. Computation of “2D complex-valued profile of wave illuminating theSLM.”

This step involves development of detailed wave optics models forcomponents that lie in the pathway between the point light source andthe SLM, and propagating the light wave emitted by the point lightsource and passing it through the related components. Aberrationsintroduced by components between the point light source and the SLM areincorporated in the computations in this manner.

4. Computation of “2D ideal analog complex-valued SLM transmittance.”

The SLM ideally should act as a multiplicative optical mask thattransforms “2D complex-valued profile of wave illuminating the SLM”computed in step 3 to the “ideal 2D complex-valued profile of wave atthe exit of SLM” computed in step 2. Thus, “2D ideal analogcomplex-valued SLM transmittance” can be found by dividing the “ideal 2Dcomplex-valued profile of wave at the exit of SLM” computed in step 2 bythe “2D complex-valued profile of wave illuminating the SLM” computed instep 3, under the assumption that the latter wave does not vanish at anypoint on the SLM. Therefore,

$\begin{matrix}{{U_{SLM}\left( {x,y} \right)} = \frac{U_{ExS}\left( {x,y} \right)}{U_{ill}\left( {x,y} \right)}} & (8)\end{matrix}$

where:

U_(SLM)(x, y) is the 2D ideal analog complex-valued SLM transmittance,

U_(ExS)(x, y) is the ideal 2D complex-valued profile of wave at the exitof SLM, and

U_(ill)(x, y) is the 2D complex-valued profile of wave illuminating theSLM.

If the SLM had submicron pixels and were full complex, the “2D idealanalog complex-valued SLM transmittance” could directly be sampled anddisplayed on the SLM. However, the restrictions on the SLM necessitatefurther processing.

5. Computation of “2D ideal complex-valued discrete SLM image.”

The finite pixel pitch of the SLM is associated with sampling of the “2Dideal analog complex-valued SLM transmittance”. In order to avoidaliasing, the bandwidth of the “2D ideal analog complex-valued SLMtransmittance” over any small region of the SLM must not exceed theinverse of SLM pixel pitch. Under the assumption that this condition issatisfied, the “2D ideal analog complex-valued SLM transmittance” can besafely sampled. However, the actual SLM has a finite aperture function,which is the interpolating function for the sampled discrete imageimplemented on the SLM. Since a non-impulsive interpolating functionimposes variations over the frequency spectrum of the interpolateddigital image, a compensation prefilter is applied to the “2D idealanalog complex-valued SLM transmittance” prior to discretization. Hence,“2D ideal complex-valued discrete SLM image” is obtained by firstapplying a compensation prefilter and then sampling the “2D ideal analogcomplex-valued SLM transmittance”.

6. Computation of “2D actual digital SLM image.”

As described above, real SLMs mostly perform some restricted type ofmodulation—such as phase only, amplitude only, binary, etc. Moreover,each pixel of a real SLM usually has a finite number of availablevalues. Therefore, the “2D ideal complex-valued discrete SLM image”needs to be quantized and encoded into a restricted type image that issuitable for displaying on the SLM. The quantization and encodingprocedure will inevitably generate some noise beams. However, it ispossible to perform the mentioned encoding and quantization such thatthe resulting noise beam falls outside the useful portion as much aspossible. This may be performed using any suitable method, including,but not limited to, Iterative Fourier Transform Algorithm,Gerschberg-Saxton algorithm, Fineup Algorithm with Don't Care, ErrorDiffusion, Direct Binary Search, etc. —These are all existing commonlyknown and used algorithms. In particular, when the noise generated as aresult of encoding is distributed in spatial frequency domain to regionsthat are outside the support of the “2D ideal complex-valued discreteSLM image”, the noise beams, after getting generated by the SLM planeand propagate to the exit pupil plane, get distributed outside theuseful portion.

Computations for embodiments that include moving SLM bar (e.g., FIGS.40, 42) are the same with the exception that the SLM is partitioned intoa number of slices, and the entire “2D actual digital SLM image” isdisplayed slice by slice in a time sequential manner depending on thescan location of the SLM bar. The scan is completed in the frame timereserved for the “2D actual digital SLM image”.

FIGS. 57-64 show a number of space-angle (or space-frequency)distributions that illustrate the basics of the computation procedure.Space angle distributions are a well-known concept in the literature andillustrate the distribution of rays as a function of transverse spatialposition over various planes in an optical system, and provide insight.In FIGS. 57-64, it is assumed that the SLM is illuminated by aconverging spherical wave obtained from a point light source and apositive lens combination, as in FIG. 12. For sake of simplicity, a fullcomplex SLM with impulsive pixels and a 2D space is assumed, where x isassumed to denote the transverse coordinates and z denotes thelongitudinal propagation axis. The angle of each ray is measured from zaxis towards +x axis.

In FIG. 57, the typical space-angle distribution of the “2Dcomplex-valued profile of virtual-scene wave on useful portion” on theexit pupil plane is illustrated, where it is assumed that the pupil iscentered around the optical axis. Note that the spatial extent is givenby the size of the useful portion, and the angular extent is given bythe desired field of view, as expected.

In FIG. 58, the space angle distribution of the “ideal 2D complex-valuedprofile of wave at the exit of SLM” is illustrated. The distributionhere is essentially equal to the horizontally sheared version of thedistribution in FIG. 57 as a result of the free-space-propagationrelation in between.

Next, in FIG. 59, the space angle distribution of the “2D complex-valuedprofile of wave illuminating the SLM” is illustrated, under theassumption that the lens in FIG. 12 forms a perfect converging sphericalwave with no optical aberrations. Then, in FIG. 60, the space angledistribution of the “2D ideal analog complex-valued SLM transmittance”,which is obtained by dividing the “ideal 2D complex-valued profile ofwave at the exit of SLM” to the “2D complex-valued profile of waveilluminating the SLM”, is illustrated. It is shown that the requiredminimum size of the SLM is determined by the FOV of the display, and therequired minimum pixel rate is determined by the width of the usefulportion.

When the “2D ideal analog complex-valued SLM transmittance” is sampledto obtain the “2D ideal complex-valued discrete SLM image,” the spacedistribution in FIG. 61 is obtained, where the distribution in FIG. 60is replicated in the angular direction as a result of sampling. FIG. 62shows the final distribution that is obtained on the exit pupil plane.Since the pixel pitch of the SLM is sufficiently small, higher orderreplicas have distributions that fall outside the spatial extent of theuseful portion.

In FIG. 63, the space angle distribution of the “2D complex-valuedprofile of wave illuminating the SLM” is illustrated, under theassumption of a more realistic behavior for the lens in FIG. 12 thatforms a converging spherical wave with spherical aberrations. Then, thespace angle distribution of the “2D ideal analog complex-valued SLMtransmittance” is obtained as illustrated in FIG. 64, where it is seenthat there is some bending around the edges of the distribution tocompensate for the presence of spherical aberration within theconverging wave provided to the SLM. Since the overall distribution atany point fits within a band that is smaller than the pixel rate of theSLM, the aberration of the lens in FIG. 12 has no severe consequencesand is automatically handled by computing the “2D ideal complex-valueddiscrete SLM image” from the space distribution in FIG. 64 instead ofthe distribution in FIG. 60.

FIG. 65 shows a method for fast hologram computation. The computationalsteps detailed above in general comprise wave propagation simulationsthat require large storage memory and high computational power. However,in some embodiments of the present invention, there are certainmathematical relations that enable the skipping of some of the steps ofthe general method. One such case is that of embodiments in which theSLM is placed directly in front of the eye and is illuminated by aconverging spherical wave. In that case, the mathematical relationbetween the “2D ideal analog complex-valued SLM transmittance” and eachplanar cross section of a virtual scene becomes simpler and a directone. To see this, let us first assume that the converging illuminationwave is generated by a collimated beam of light and a thin positive lenswith a focal length given by D (the distance between SLM and eye), thatis placed immediately before the SLM. Second, we should note that theorders of the SLM and the lens can be changed, since both are assumed tobe thin multiplicative components. As a result, we get an equivalentsystem in which collimated light illuminates the SLM and then passesthrough an eyepiece lens to get directed towards the eye. Finally,consider the virtual scene shown in FIG. 65, in particular, the planarcross section labeled as Plane 4. Each point on this planar crosssection, which is assumed to be quite far away from the eye, would sendan almost parallel bundle of rays to the useful portion. These rays,when traced backwards to the above-mentioned eyepiece lens and passedbackwards through it, get focused on the plane situated at d_(eq4), andform a demagnified image of the planar cross section named Plane 4.Hence, the portion of the “2D ideal analog complex-valued SLMtransmittance” responsible from Plane 4 of the virtual scene is actuallyequivalent to the diffraction pattern of the demagnified image of plane4 that lies on d_(eq4). The discussion applies similarly to other planarcross sections of the virtual scene. As seen in FIG. 65, the images ofthe planar cross sections all have the same size, and each point ofthese images only send a narrow cone of rays to the SLM surface that isalmost parallel to the optical axis. Hence, the computation procedurecan be carried out with a larger step size, and with simulation windowsof a common and smaller size, lowering the memory requirementsdrastically. Also, the computation procedure for each planar crosssection is completed in parallel through the well-known Angular Spectrummethod, hence no loops over the points on a planar cross section areneeded.

Accordingly, the two-dimensional complex-valued profile of avirtual-scene wave on a useful portion of an exit pupil plane may insome embodiments be determined by:

partitioning the virtual scene into a plurality of spherical surfacesconcentric at the center of the useful portion of the exit pupil planewith different radius;

forming a matrix for each of the spherical surfaces where each elementof the matrix is associated with a specific angular location on thesphere, and each element is filled with the complex amplitude of thepoint source at that angular location on the sphere;

inverse Fourier transforming the matrix to create a result;

multiplying the result by a common diverging lens term with a focallength equal to the radius of the sphere; and

repeating the partitioning, forming, inverse Fourier transforming, andmultiplying for each of the plurality of spherical surfaces andsuperposing to find the two-dimensional complex-valued profile of thevirtual-scene wave on the useful portion of the exit pupil plane.

FIG. 66 illustrates the fundamentals of the method for deliveringspeckle free images to the retina of a user. Speckle noise is observedwhen real objects are imaged by a user under coherent light, due to thefact that surface irregularities of real objects lead to a random phasedistribution over the object. In particular, each point of a real objectpoint is imaged as an Airy disk on the retina due to diffraction fromeye pupil. Hence, individual images of object points overlap on theretina. Since the illumination is coherent, these individual images dointerfere with each other. The random phase on the real object pointscauses this interference pattern to exhibit rapid constructive anddestructive interference patterns, which are seen as the speckle noiseover the image of a real object. It is also well known that specklenoise depends on the relative position between eyes of an observer andthe object that is imaged, and the speckle noise pattern changes as theeye moves.

The various embodiments of the present invention are not imaging realobjects under coherent illumination. Rather, they are delivering imagesof virtual objects to the retina using coherent illumination. This caseis different than the case of real objects in that we have total controlover the phase distribution that we can assign to virtual object points.

In FIG. 66, the image on the retina of a virtual object that consists ofa number of point sources is illustrated for two cases. In both cases,the final continuous intensity function that forms on the retina isobtained as an interpolated version of the discrete images of virtualobject points. In fact, assuming a rectangular eye pupil with dimensionswp_(x) and wp_(y) and centered around (x_(p), y_(p)), and a planarvirtual object placed at a distance d from the eye that consists of Mpoint sources located at (x_(i), y_(i)) on the transverse plane, theeffective intensity distribution as seen by the eye becomes:

$\begin{matrix}{{{I_{EO}\left( {x,y} \right)} = {{\sum\limits_{i = 1}^{M}{c_{i}e^{J\frac{2\pi}{\lambda}R_{i}}\sin \; {c\left( \frac{x - x_{i}}{\left( \frac{\lambda \; d}{{wp}_{x}} \right)} \right)}\sin \; {c\left( \frac{y - y_{i}}{\left( \frac{\lambda \; d}{{wp}_{y}} \right)} \right)}}}}^{2}}{{where}\text{:}}} & (9) \\{{{\sin \; {c(x)}} = \frac{\sin \left( {\pi \; x} \right)}{\pi \; x}},} & (10) \\{{R_{i} = \sqrt{\left( {x_{i} - x_{p}} \right)^{2} + \left( {y_{i} - y_{p}} \right)^{2} + d^{2}}},} & (11)\end{matrix}$

I_(EO)(x, y) is the intensity of effective object that is seen by theeye at the current position of eye pupil,

c_(i) is the complex amplitude of the i^(th) object point,

(wp_(x), wp_(y)) are the dimensions of the eye pupil—assumed to berectangular here, and

(x_(i), y_(i)) are the transverse coordinates of each object point.

In the upper case, a random phase variation is assigned to the objectpoints. As a result, the intensity function exhibits rapid intensityvariations between the discrete images of virtual object points. Theuser perceives these rapid changes as the speckle noise. In the lowercase, an appropriate phase distribution has been assigned to the virtualobject points. As a result, the intensity function that forms on theretina is a smoothly interpolated version of discrete images of virtualobject points. Hence, the image delivered by the system resembles theimage that would be seen under incoherent illumination, free of specklenoise.

In particular, if the light from each of the individual virtual objectpoints arrives at the retina of the user with the same phase, then theinterpolation that forms on the retina becomes smooth. An equivalentcondition is that light waves from each object point to arrive at thepupil of the user in phase. Therefore, if a virtual object point thathas a distance of R to the center of the pupil is assigned a phase ofe^(−jkR) with k denoting the wave number, the light from all virtualobject points arrives on the pupil of the user in phase, and form aspeckle-free retinal image. Note that the proposed phase assignment isspecific to a certain pupil position and wavelength. Hence, it must beupdated when the pupil location changes, and when the object wave withinthe useful portion is calculated for a different color.

To sum up, in the embodiments of this invention, during the computationof the “2D complex-valued profile of virtual-scene wave on usefulportion” (see FIG. 66); the phase assignment rule explained here isused. In this manner, the virtual objects are imaged free of speckle.

Various embodiments of Back-Light Units (BLUs) are now described. Manyof the BLUs described below are suitable for use in an illuminationoptics module such as illumination optics module 440 (FIG. 4). VariousBLU embodiments create coherent light beams that may be converging,diverging, or collimated. BLUs are also described as being part ofnear-to-eye display devices. The BLUs may be incorporated in anynear-to-eye described herein, including for example those described withreference to FIGS. 1, 3, 35, and 53.

FIG. 67 shows a perspective view of a back-light unit that generates atwo-dimensional converging beam. The rays emanate from a transparentsubstrate and focus on the convergence point. Back-light unit 6700includes first face 6710 from which a converging light beam emanates.Back-light unit 6700 also includes second face 6720. In someembodiments, faces 6710 and 6720 are parallel, but this is not alimitation of the present invention.

Apparatus 6700 is referred to as a “back-light unit” in part because itcan be used to “back light” an SLM with a converging beam (or other typeof beam). Optically, back-light unit 6700 is equivalent to thecombination of point light source 120 and lens 1210 as shown in FIG. 12;however, back-light unit 6700 provides a significant space savings ascompared to the system shown in FIG. 12.

FIG. 68 shows a cross section of a back-light unit. Back-light unit 6800corresponds to back-light unit 6700 (FIG. 67) implemented with alight-scattering apparatus 6830 and a reflective optical elementarranged as a planar micromirror array 6810. The term “planarmicromirror array” as used herein refers to the individual mirrors beingarranged on a plane, and is not meant to infer that each mirror has thesame tilt angle. The light emanating from light-scattering apparatus6830 hits the micromirror array and then focuses on the convergencepoint. The position of each individual micromirror in the array 6810 isarranged such that it reflects the incoming ray from light-scatteringapparatus 6830 to the convergence point. In order to have a transparentsubstrate, the micromirror array 6810 is buried in a refractive indexmatched medium. In some embodiments, the reflectivity of the micromirrorarray can be provided either by notch coating, semi-reflective thinmetal coating, or the like.

Light-scattering apparatus 6830 scatters light away from the first face6710, and micromirror array 6810 reflects the light from scatteringapparatus 6830 to first face 6710 and creates the converging light beam.In some embodiments, light-scattering apparatus 6830 receives light froman external light source (not shown), and in other embodiments,light-scattering apparatus 6830 is co-located with one or more lightsources embedded within the back-light unit, and scatters light awayfrom the first face 6710. For example, in some embodiments, an organiclight emitting diode (OLED) is embedded within the substrate to providelight to the light-scattering apparatus 6830. Also for example, in someembodiments, red, green, and blue (RGB) OLEDs are included in back-lightunit 6800. Further, in some embodiments, a fluorescent molecule such asa quantum dot is embedded in the substrate as a light source. Ingeneral, any of the back-light units described herein may include anyinternal or external light source.

In some embodiments, the light-scattering apparatus 6830 includes adiffusive material such as silver epoxy or epoxy with embeddedmicroparticles. Further, in some embodiments, the same scatteringapparatus is provided for all the colors. Some embodiments includemultiple scattering apparatus (a “source array”) in order to increaseFOV.

FIG. 69 shows a cross section of a back-light unit. Back-light unit 6900corresponds to back-light unit 6700 (FIG. 67) implemented withlight-scattering apparatus 6830 and reflective optical element 6910arranged as a Fresnel mirror. Light-scattering apparatus 6830 scatterslight away from the first face 6710, and Fresnel mirror 6910 reflectsthe light from scattering apparatus 6830 to first face 6710 and createsthe converging light beam.

FIG. 70 shows a cross section of a back-light unit. Back-light unit 7000corresponds to back-light unit 6700 (FIG. 67) implemented withlight-scattering apparatus 6830 and a reflective optical elementarranged as a free-form concave reflector 7010. Light-scatteringapparatus 6830 scatters light away from the first face 6710, andreflector 7010 reflects the light from scattering apparatus 6830 tofirst face 6710 and creates the converging light beam.

FIG. 71 shows a cross section of a back-light unit. Back-light unit 7100corresponds to back-light unit 6700 (FIG. 67) implemented withlight-scattering apparatus 6830 and a reflective optical elementarranged as a nonplanar micromirror array 7110. Light-scatteringapparatus 6830 scatters light away from the first face 6710, andnonplanar micromirror array 7110 reflects the light from scatteringapparatus 6830 to first face 6710 and creates the converging light beam.Nonplanar micromirror array 7110 reduces the shadowing effects inbetween the individual mirrors of a planar micromirror array.

FIG. 72 shows a cross section of a back-light unit and an external pointlight source. Back-light unit 7200 corresponds to back-light unit 6700(FIG. 67) implemented with light-scattering apparatus 6830 and areflective optical element arranged as a planar micromirror array 6810.Light-scattering apparatus 6830 scatters light away from the first face6710, and planar micromirror array 6810 reflects the light fromscattering apparatus 6830 to first face 6710 and creates the converginglight beam. The light emanates from an external point light source 120,and is focused on light-scattering apparatus 6830 inside the transparentsubstrate.

A combination of an SLM and a transparent back-light unit withconverging beam output can be used as a near-to-eye display device. FIG.73 shows a near-to-eye display device that includes back-light unit 7300and transmissive SLM 410. Back-light unit 7300 corresponds to back-lightunit 6700 (FIG. 67) implemented with light-scattering apparatus 6830 anda reflective optical element arranged as a planar micromirror array6810. Light-scattering apparatus 6830 scatters light away from the firstface 6710, and micromirror array 6810 reflects the light from scatteringapparatus 6830 to first face 6710 to create the converging light beam.The converging beam at the output of the back-light unit passes throughtransmissive SLM 410 and then focuses on the eye pupil. In thisconfiguration, the SLM has a computer-generated hologram written on itin order to construct the desired light field on the retina.

Alternatively, a reflective SLM 110 can be used in the near-to-eyedisplay device instead of the transparent SLM as can be seen in FIG. 74.FIG. 74 shows a near-to-eye display device that includes back-light unit7400 and reflective SLM 110. Back-light unit 7400 corresponds toback-light unit 6700 (FIG. 67) implemented with light-scatteringapparatus 6830 and a linearly arranged transreflective micromirror array7410. Light-scattering apparatus 6830 scatters light away from the firstface 6710, and transreflective micromirror array 7410 reflects the lightfrom scattering apparatus 6830 to first face 6710 where it is modulatedand reflected by reflective SLM 110. The modulated virtual-scene wavepasses back through transreflective micromirror array 7410 and emanatesfrom the second face 6720 as a converging beam that focuses on the eyepupil. In this configuration, the SLM has a computer-generated hologramwritten on it in order to construct the desired light field on theretina.

The light scattered in the direction of the convergence point from thelight-scattering apparatus can create a bright spot on the retina innear-to-eye display device applications. This unwanted portion of thescattered light can be blocked by using cross polarizers between thelight-scattering apparatus and the convergence point as can be seen inFIG. 75. Back-light unit 7500 corresponds to back-light unit 6700 (FIG.67) implemented with light-scattering apparatus 6830 and micromirrorarray 6810. Light-scattering apparatus 6830 scatters light away from thefirst face 6710, and micromirror array 6810 reflects the light fromscattering apparatus 6830 to first face 6710 and creates the converginglight beam. Back-light unit 7500 also includes cross polarizers 7510. Insome embodiments, cross polarizers 7510 are two orthogonally polarizedoptical elements to block the passage of light. When cross polarizers7510 are included, the bright spot referred to above is not present onthe retina.

Alternatively, a buried curved mirror, which reflects the rays back tothe scattering apparatus, can be used instead of cross polarizers as canbe seen in FIG. 76, making it more light efficient. FIG. 76 shows across section of a back-light unit. Back-light unit 7600 corresponds toback-light unit 6700 (FIG. 67) implemented with light-scatteringapparatus 6830 and a reflective optical element arranged as a planarmicromirror array 6810. Light-scattering apparatus 6830 scatters lightaway from the first face 6710, and planar micromirror array 6810reflects the light from scattering apparatus 6830 to first face 6710 andcreate the converging light beam. Back-light unit 7600 also includesmirror 7610. Mirror 7610 blocks light reflected from micromirror array6810 that would otherwise create a bright spot on the retina. Whenmirror 7610 is included, the bright spot referred to above is notpresent on the retina.

FIG. 77 shows a cross section of a back-light unit with a fiber. Thelight carried by the fiber hits the 45°-angled mirror 7730 and isdirected to light-scattering apparatus 6830, which is used forincreasing the solid angle of the ray bundle for fully covering themicromirror array. Light-scattering apparatus 6830 scatters the lightaway from first face 6710 and towards the micromirror array 6810. Thescattered light is then reflected off the micromirror array 6810 andemanates from first face 6710 as a converging beam.

In some embodiments, light-scattering apparatus 6830 can be realized byusing high refractive index transparent nanoparticles. One advantage ofthis system can be explained as follows: the different colors can becoupled into the same fiber and directed to the same scatteringapparatus. Therefore, the effective positions of the different-coloredlight sources do not change with respect to the micromirror array, whichreduces the chromatic aberrations. In some embodiments, the end face offiber 7710 is polished with a 45° angle and coated with metal, which isused instead of mirror 7730.

The back-light unit can be arranged such that the output beam has aprofile different than the converging beam. For example, by arrangingthe position of the individual mirrors in the micromirror array, aone-dimensional converging beam can be generated as shown in FIG. 78.Similarly, collimated and diverging beams can be generated as can beseen in FIG. 79 and FIG. 80, respectively.

Various embodiments of wedge-based back-light units are now described.Many of the wedge-based back-light units may be used in illuminationoptics modules such as illumination optics module 440 (FIG. 4).Wedge-based back-light units are also described as being part ofnear-to-eye display devices. The wedge-based back-light units may beincorporated in any near-to-eye display device described herein,including for example those described with reference to FIGS. 1, 3, 35,and 53.

FIG. 81 shows a cross section of a slab waveguide, a wedge, and acomponent with a micromirror array. Apparatus 8100 includes slabwaveguide 8110, wedge 8120, and component 8130 with micromirror array8132. Slab waveguide 8110 includes input end 8112, output end 8114,first surface 8118, and second surface 8116. First surface 8118 andsecond surface 8116 are parallel to each other to cause light topropagate from input end 8112 to output end 8114 by total internalreflection.

Wedge 8120 is coupled to the output end 8114 of slab waveguide 8110.Wedge 8120 includes first surface 8128 and slanted surface 8126 that arenot parallel to each other. First surface 8128 and slanted surface 8126form a continuously decreasing thickness to cause light received fromslab waveguide 8110 to exit wedge 8120 from slanted surface 8126. Insome embodiments, first surface 8128 is parallel to first surface 8118,and in other embodiments, slanted surface 8126 is parallel to firstsurface 8118.

Optical component 8130 includes face 8138 that is oriented parallel toslanted surface 8126. Further, optical component 8130 includesmicromirror array 8132 to reflect light received through face 8138 backout through the same face 8138 and through wedge 8120. Micromirror array8132 may be any type of micromirror array including but not limited tothose shown in, and described with reference to, FIGS. 68-76.

In some embodiments, optical component 8130 has a shape that performs asa compensating wedge for see-through capability. In these embodiments,optical component 8130 is referred to a compensating wedge. Whenfunctioning as a compensating wedge, optical component 8130 has a wedgeshape that complements the shape of wedge 8120 such that light travelingthrough both the wedge and compensating wedge travel through the sameamount of material. This eliminates any prism effect that wouldotherwise be perceived by a user. Optical component 8130 is positionedto provide a uniform air gap 8140 between slanted surface 8126 and face8138. In embodiments with semitransparent micromirror arrays,undistorted real-world views are provided because of the combination ofthe wedge and optical component 8130 in shape of a compensating wedge.

In operation, point light source 120 creates a diverging light beam. Thediverging light beam enters slab waveguide 8110 at input end 8112 andpropagates by total internal reflection within slab waveguide 8110 tooutput end 8114, at which point it enters wedge 8120. As the light beampropagates in wedge 8120, the internal angle of incidence changes due tothe decreasing thickness, allowing the light beam to exit almostcollimated from the slanted surface 8126 of wedge 8120. The light thenenters into optical component 8130 at face 8138 and hits micromirrorarray 8132. The light reflected from the micromirror array goes throughwedge 8120, exiting at surface 8128 as a converging wave, and thenfocuses onto exit pupil plane 220.

FIG. 82 shows a top view of the apparatus of FIG. 81. The light enteringinto the slab waveguide 8110 expands in the lateral direction and isconfined in the vertical direction by total internal reflection. Thenthe light enters into the wedge region and the rays start to exit fromthe wedge since the incidence angles are reduced at each reflection.

FIGS. 83-88 combine wedge-based back-light units with SLMs to formnear-to-eye display devices. In operation, these perform the functionsof both illumination optics module 440 and SLM 410 (FIG. 4). Directapplications to near-to-eye display devices are also described.

FIG. 83 shows a cross section of a slab waveguide, wedge, opticalcomponent with micromirror array, and SLM positioned along the slabwaveguide. In this configuration, the light field hits reflective SLM110 while it is propagating in slab waveguide 8110. Although SLM 110 isshown on surface 8118 of slab waveguide 8110 in FIG. 83, this is not alimitation of the present invention. In some embodiments, the SLM isplaced on surface 8116. The computer-generated hologram on the SLMmodulates the light as it propagates in slab waveguide 8110, and thedesired virtual scene is generated at the useful portion of exit pupilplane 220 as described above.

FIG. 84 shows a cross section of a slab waveguide, a wedge, a componentwith a micromirror array, and an SLM between the wedge and the componentwith the micromirror array. A transmissive SLM 410 is placed in betweenwedge 8120 and optical component 8130. In order to generate the desiredlight field at the exit pupil plane 220, the collimated light at theoutput of wedge 8120 passes through transmissive SLM 410 which has acomputer-generated hologram on it, and hits micromirror array 8132. Thelight field reflects from micromirror array 8132, passes throughtransmissive SLM 410 again and then converges on exit pupil plane 220.The light that enters from the eye pupil then constructs the desiredcontent on the retina.

FIG. 85 shows a cross section of a slab waveguide, wedge, component witha micromirror array, and an SLM below the wedge. FIG. 85 is similar toFIG. 83 except that the SLM is below the wedge and it is transmissive.The combination of FIG. 85 can be used as a near-to-eye display device.

FIG. 86 shows a cross section of a slab waveguide, wedge, component withmicromirror array, and an SLM at entrance to the slab. FIG. 86 issimilar to FIG. 85 except that the SLM is at the input end of the slabwaveguide. The combination of FIG. 86 can be used as a near-to-eyedisplay device.

FIG. 87 shows a cross section of a slab waveguide, wedge, compensatingwedge with micromirror array, and SLM below the wedge. In embodimentsrepresented by FIG. 87, SLM 110 is reflective, and micromirror array8732 is transreflective. Light first exiting wedge 8120 enters opticalcomponent 8730 and is reflected off micromirror array 8732 as aconverging beam. The converging beam then passes back through wedge 8120to be reflected and modulated by reflective SLM 110. The light reflectedoff SLM 110 passes back through optical component 8730, and converges atthe exit pupil plane 220.

FIG. 88 shows a cross section of slab waveguide with a 90-degree bend,wedge, optical component with a micromirror array, and an SLM. Thenear-to-eye display device of FIG. 88 is similar to the near-to-eyedisplay device of FIG. 85 with the exception that slab waveguide 8810includes a 90-degree bend in FIG. 88. Light rays propagating in slabwaveguide 8810 couple into wedge 8120 by means of a turning mirror 8820.Placing at least a portion of the slab waveguide perpendicular to themajor axis of the wedge as shown in FIG. 88 can reduce the form factorof the wedge-based near-to-eye display device.

A wedge-based eye tracker can be constructed as can be seen in FIG. 89.A near-infrared (NIR) illumination provided by light source 8950 iscoupled into the slab after passing through a beam splitter 8910 and therays exit from the wedge. A light turning film 8940 is placed on thewedge for directing the light beam towards the eye. The light reflectedback from the eye is coupled back into the wedge 8120 and forms theimage of the eye onto the camera, which can be used for eye tracking.

Camera 8930 is shown at the input to slab waveguide 8110 and coupledwith a beamsplitter 8910. In some embodiments, camera 8930 is positionedalong slab waveguide on either surface 8116 or 8118 similar to SLM 110in FIG. 83.

FIG. 90 shows a near-to-eye display device with a slab waveguide, wedge,component with micromirror array, SLM, and camera for eye tracking.Near-to-eye display device 9000 is in the form of a head-worn device,and more specifically in the shape of a pair of eyeglasses, but this isnot a limitation of the present invention. In some embodiments,near-to-eye display device 9000 is a handheld device, and in otherembodiments, near-to-eye display device 9000 is a fixed device that auser rests against to create a constant eye relief.

Near-to-eye display device 9000 includes slab waveguides 8810, wedges8120, optical components 8130, optical components 9010, cameras 9020,and light sources 120. Near-to-eye display device 9000 also showsreflective SLM 110 on the slab waveguide 8810, although this is not alimitation of the present invention. Any SLM, either transmissive orreflective may be positioned anywhere as shown above in the previousfigures without departing from the scope of the present invention. Forexample, in some embodiments, a reflective SLM is placed in opticalcomponent 9010, and in other embodiments, a transmissive SLM is placedat display area 9030.

In some embodiments, near-to-eye display device 9000 is anaugmented-reality device that allows real-world light to pass throughoptical components 9010, 8130, and wedge 8120. In these embodiments, thereal-world view is superimposed on any virtual scene created by thenear-to-eye display device to create an augmented reality for the userof near-to-eye display device 9000.

In some embodiments, near-to-eye display device 9000 includeselectronics to provide SLM data to the SLMs. The electronics may includea processor and memory, or may include cabling and transmission circuitsto receive data from external sources. The manner in which data isprovided to the SLMs is not a limitation of the present invention.

FIG. 91 shows a slab waveguide, a curved wedge and a compensation plate.Apparatus 9100 includes slab waveguide 8110, curved wedge 9120, andcurved compensation plate 9130. Slab waveguide 8110 includes input end8112, output end 8114, first surface 8118, and second surface 8116. Asdescribed above with reference to FIG. 81, first surface 8118 and secondsurface 8116 are parallel to each other to cause light to propagate frominput end 8112 to output end 8114 by total internal reflection.

Curved wedge 9120 is coupled to the output end 8114 of slab waveguide8110. Curved wedge 9120 includes first curved surface 9128 and secondcurved surface 9126 that form a continuously decreasing thickness. Insome embodiments, curved wedge is constructed from a refractive materialhaving a graded refractive index (GRIN). The curvature of curved wedge9120 and the gradient of the refractive index in the GRIN material areselected such that light received from slab waveguide 8110 exits curvedwedge 9120 from curved surface 9128 as a converging beam that focuses onexit pupil plane 220.

Compensating wedge 9130 includes surface 9138 having substantially thesame curvature as surface 9126, and is positioned to provide a uniformair gap 9140 between curved surface 9126 and surface 9138. Compensatingwedge 9130 has a wedge shape that complements the shape of wedge 9120such that light traveling through both the curved wedge and thecompensating wedge travel through an equivalent amount oflike-refractive material. This eliminates any prism effect that wouldotherwise be perceived by a user. Undistorted real-world views areprovided because of the combination of the curved wedge and compensatingwedge 9130.

In operation, a light beam enters slab waveguide 8110 at input end 8112and propagates by total internal reflection within slab waveguide 8110to output end 8114, at which point it enters wedge 9120. As the lightbeam propagates in curved wedge 9120, the internal angle of incidencechanges due to the decreasing thickness, and the critical angle changesdue to the graded refractive index, allowing the light beam to exitcurved surface 9128 of curved wedge 9120 as a converging wave thatfocuses onto exit pupil plane 220.

FIG. 92 shows a slab waveguide, curved wedge, and SLM in a convergingbeam. Apparatus 9200 includes slab waveguide 8110, curved wedge 9120 andtransmissive SLM 410. Transmissive SLM 410 is placed in the convergingbeam path and modulates the beam to create a virtual-scene light wavedistribution on exit pupil plane 220. Apparatus 92 may also include acompensating wedge such as compensating wedge 9130 (FIG. 91).

FIG. 93 shows a slab waveguide, curved wedge, and SLM on top of theslab. In this configuration, the light field hits reflective SLM 110while it is propagating in slab waveguide 8110. Although SLM 110 isshown on surface 8116 of slab waveguide 8110 in FIG. 93, this is not alimitation of the present invention. In some embodiments, the SLM isplaced on surface 8118. The computer-generated hologram on the SLMmodulates the light as it propagates in slab waveguide 8110, and thedesired virtual scene is generated at the useful portion of exit pupilplane 220 as described above.

FIG. 94 shows a slab waveguide, curved wedge, and SLM at the entrance tothe slab waveguide. FIG. 94 is similar to FIG. 93 except that SLM 410 isat the input end of the slab waveguide, and SLM 410 is transmissive.

FIG. 95 shows a slab waveguide, curved wedge, and camera for eyetracking. An NIR illumination provided by light source 8950 is coupledinto the slab after passing through a beam splitter 8910. The operationis similar to that described with respect to FIG. 89 in which the lightreflected back from the eye is coupled back into the wedge 9120 andforms the image of the eye onto the camera, which can be used for eyetracking.

Camera 8930 is shown at the input to slab waveguide 8110 and coupledwith a beamsplitter 8910. In some embodiments, camera 8930 is positionedalong slab waveguide on either surface 8116 or 8118 similar to SLM 110in FIG. 83.

FIG. 96 shows a perspective view of the apparatus of FIG. 91. FIG. 91shows light representing a real-world view passing through bothcompensating wedge 9130 and curved wedge 9210. The real-world view maybe superimposed on any modulated light distribution and presented at theexit pupil plane to form an augmented-reality display.

FIG. 97 shows a near-to-eye display device with a slab waveguide, curvedwedge, compensating wedge, SLM, and camera for eye tracking. Near-to-eyedisplay device 9700 is in the form of a head-worn device, and morespecifically in the shape of a pair of eyeglasses, but this is not alimitation of the present invention. In some embodiments, near-to-eyedisplay device 9700 is a handheld device, and in other embodiments,near-to-eye display device 9700 is a fixed device that a user restsagainst to create a constant eye relief.

Near-to-eye display device 9700 includes slab waveguides 8810, curvedwedges 9120, compensating wedges 9130, cameras 9020, and light sources120. Near-to-eye display device 9700 is shown with slab waveguides 8810including a 90-degree bend as described above with reference to FIGS. 88and 93. Near-to-eye display device 9700 also shows reflective SLM 110 onthe slab waveguide 8810, although this is not a limitation of thepresent invention. Any SLM, either transmissive or reflective may bepositioned anywhere as shown above in the previous figures withoutdeparting from the scope of the present invention.

In some embodiments, near-to-eye display device 9700 is avirtual-reality device that blocks the real-world view and provides theuser with a virtual scene at the useful portion of the exit pupil plane.In other embodiments, near-to-eye display device 9700 is anaugmented-reality device that allows real-world light to pass throughthe compensating wedge 9130 and the curved wedge 9120. In theseembodiments, the real-world view is superimposed on any virtual scenecreated by the near-to-eye display device to create an augmented realityfor the user of near-to-eye display device 9700.

In some embodiments, near-to-eye display device 9700 includeselectronics to provide SLM data to the SLMs. The electronics may includea processor and memory, or may include cabling and transmission circuitsto receive data from external sources. The manner in which data isprovided to the SLMs is not a limitation of the present invention.

Various embodiments of moving platform based near-to-eye display devicesare now described. FIG. 98 shows a near-to-eye display device with amoving platform assembly. Near-to-eye display device 9800 includesmoving platform assembly 9802 and electronics module 160. Near-to-eyedisplay device 9800 may include many more components such as wiring,cabling, camera, and the like. These components are intentionallyomitted for clarity. In addition, near-to-eye display device 9800 isshown with a moving platform assembly 9802 on only one side, whereas inpractice, near-to-eye display device 9800 may have two moving barassemblies 9802—one on each side.

Moving platform assembly 9802 includes moving platform 9804 and coils9840. Moving platform 9804 includes LED array 9810, LED drivers 9820,and magnets 9830 for actuation. LED drivers 9820 may be integratedcircuits affixed to moving platform 9804. LED drivers 9820 causeindividual LEDs in LED array 9810 to be illuminated in response toelectrical signals received from electronics module 160. In someembodiments, LED array 9810 may be a one-dimensional array of red,green, and blue LEDs. For example, LED array 9810 may include one row ofred LEDs, one row of green LEDs, and one row of blue LEDs. In otherembodiments, LED array 9810 may be a two-dimensional array of red,green, and blue LEDs. For example, LED array 9810 may include multiplerows of red LEDs, multiple rows of green LEDs, and multiple rows of blueLEDs.

In operation, moving platform 9804 moves vertically across a user'sfield of view. Moving platform 9804 carries two permanent magnets 9830.Two linear arrays of electromagnetic coils 9840 are attached to themoving platform assembly 9802 outside the display area. Current can bepassed through any given subset of the coils 9840 to actuate movingplatform 9804. Electronics module 160 actuates moving platform 9804 anddrives LED drivers 9820 synchronously such that a transparent image iscreated for a user.

The operation of moving platform assembly 9802 effectively creates animage on a transparent screen. The area occupied by the transparentscreen is referred to herein as the “display area.”

FIG. 99 shows a cross section of moving platform assembly 9802 and apolarizing film 9910. Moving platform 9804 is shown with a cross sectionof a one-dimensional array of LEDs. Further, the actuation in thedirection of the arrows is accomplished by energizing coils 9840 insequence so that magnet 9830 is either attracted or repulsed. The timingof coil energizing is synchronous with driving the LEDs so that an imageis displayed forming an effective transparent screen for the user.

Polarizing film 9910 is oriented such that environmental light viewed bya user of near-to-eye display device 9800 passes through polarizing film9910, and further oriented such that light produced by the plurality oflight sources does not pass through the polarizing film. In someembodiments light from LED array 9810 is also polarized. In theseembodiments, light passing through the polarizer is polarized in a firstorientation and light emitted from the LEDs is polarized in a secondorientation orthogonal to the first orientation. In some embodiments,polarizing film 9910 is omitted.

FIG. 100 shows a perspective view of a moving platform assembly. Movingplatform assembly 9802 is shown with frame 11010, coils 9840 and movingplatform 9804. Frame 11010 and moving platform 9804 are showninterconnected by flex cable 11020. Flex cable 11020 carries signalsfrom electronics module 160 (FIG. 98) to LED drivers 9820 on movingplatform 9804. As shown in FIG. 100, moving platform 9804 includes onemoving bar that has an array of light sources mounted thereon.

FIG. 101 shows a side view of a contact lens placed on an eye. Contactlens 10100 includes two concentric portions, a peripheral portion 10120,and a central portion 10110. Central portion 10110 has a high-diopterlens to allow a user to focus at a plane of the plurality of lightsources on moving platform 9804 when wearing near-to-eye display device9800. Peripheral portion 10120 of the contact lens admits only lightpolarized in a first orientation, and central portion 10110 of thecontact lens admits only light polarized in a second orientation,orthogonal to the first orientation. In some embodiments, centralportion 10110 admits the polarized light emitted from LED array 9810,and peripheral portion 10120 admits the polarized light that has passedthrough polarizing film 9910.

FIG. 102 shows a front view of the contact lens of FIG. 101. FIG. 102shows three different variations of contact lens 10100. Variation A hascentral portion 10110 split into two different parts; variation B hascentral portion 10110 split into three different parts; and variation Chas central portion 10110 split into four different parts. In a givencontact lens, each different part of central portion 10110 has adifferent color filter to separate different color components of thelight emanating from the plurality of light sources.

FIG. 103 shows a cross section of a contact lens on an eye and a movingplatform assembly. Moving platform assembly 9802 includes movingplatform 9804 which carries a plurality of light sources to form atransparent display for the user. The light from the surroundings, showngenerally at 10310, is polarized in a first orientation by polarizingfilm 9910. The light from the plurality of light sources is polarized ina second orientation, orthogonal to the first orientation. Theperipheral portion 10120 of the contact lens is constructed so that itonly admits light with the first orientation. The central portion 10110of the contact lens is constructed so that it only admits light with thesecond orientation. The central portion 10110 of the contact lens issplit into multiple parts, each having a separate color filter toseparate different color components of the light emanating from theplurality of light sources.

The portion of the light from the plurality of light sources that passesthrough the high-diopter lens in the central portion 10110 of thecontact lens is properly focused in a user's eye. This allows a user tofocus at a plane of the plurality of light sources. The portion of thelight from the surroundings that passes through the outer portion 10120of the contact lens allows a user to see the surroundings with theuser's normal eye sight.

FIG. 104 shows a near-to-eye display device with a moving platformassembly. As shown in FIG. 104, near-to-eye display device 10400includes moving platform assembly 10402, which in turn includes a movingplatform with multiple bars. In operation, the multiple moving bars movevertically together across the user's field of view as the movingplatform moves. Each bar may contain a one-dimensional ortwo-dimensional array of light sources. Actuation is the same asdescribed above with reference to FIGS. 98-100.

FIG. 105 shows a perspective view of a near-to-eye display device with arotating bar. Rotating bar 10510 includes a plurality of light sourcesand rotates about pivot point 10520. Rotating bar 10510 is actuatedsynchronously with signals that drive the light source to create aneffective transparent display for the user.

FIGS. 106-108 show front views of near-to-eye display devices withrotating bars. FIG. 106 shows a front view of near-to-eye display device10500 with the detail shown for the right eye rather than the left eye.The rotating bar 10510 rotates about pivot point 10520 and sweepsthrough the display area shown at 10610. In some embodiments, therotating bar includes a plurality of light sources as described abovewith reference to previous figures.

FIG. 107 shows a front view of a near-to-eye display device with arotating bar rotating around two pivot points. The rotating bar 10710carries a plurality of light sources. A rotating arm 10720 is rotatingaround a first pivot point. The rotating arm 10720 is connected to therotating bar 10710 at the second pivot point. The rotating bar 10710 iskept at a fixed orientation throughout the motion so that the displaycan make a more efficient use of the motion. The dotted line outlinesthe potential display area.

FIG. 108 shows front view of a near-to-eye display device with a movingbar moving vertically across a user's field of view. The moving bar10830 is actuated by a rotating arm 10820 that rotates around a pivotpoint 10520. The rotating arm 10820 is attached to a groove on themoving bar 10830. The rotating arm 10820 can move along the groove. Themoving bar 10830 is constrained by two mechanical guides 10810 toproduce a vertical motion. The moving bar 10830 carries a plurality oflight sources. The dotted line outlines the potential display area.

FIG. 109 shows a rotating-bar-actuation embodiment where a permanentmagnet 10910 is placed inside of an electromagnetic coil 10930. Thepermanent magnet 10910 is attached to the rotating bar 10510 and issuspended so that there is a pivot point 10520 inside of theelectromagnetic coil 10930. When a current is passed through theelectromagnetic coil 10930, the rotating bar 10510 will rotate aroundthe pivot point 10520. A small stabilization magnet 10920 is attached tothe electromagnetic coil 10930 to keep the rotating bar 10510 stablewhen not actuated.

FIG. 110 shows a rotating-bar-actuation embodiment where a permanentmagnet 10910 is placed between two electromagnetic coils 11030. Thepermanent magnet 10910 is attached to the rotating bar 10510 and issuspended so that the rotating bar 10510 will rotate around the pivotpoint 10520. When current is passed though the electromagnetic coils11030, the rotating bar 10510 will rotate around the pivot point 10520.The various embodiments of the present invention are not limited tomagnetic actuation. For example, in some embodiments, piezoelectricactuation is employed, and in other embodiments, actuation using arotary or linear motor of any sort is employed.

FIG. 111 shows a front view of a near-to-eye display device with amoving bar that moves in two dimensions. The moving bar 11130 movesperiodically in the vertical direction to form a transparent display fora user, and it simultaneously moves periodically a shorter distance inthe horizontal direction. The purpose of the horizontal motion is toincrease the horizontal display resolution above the resolution dictatedby the spacing of the light sources.

FIG. 112 shows an external near-to-eye display device 11200 with nocontact lens. A moving bar (not shown) is moving across an otherwisetransparent area in a near-to-eye display device. The plurality of lightsources is arranged so that light from the display can reach observersother than the user of the device. If an observer views the transparentdisplay from a distance where the observer's eyes can focus on thecontent on the transparent display, the observer sees image 11210. Twoexamples of image 11210 are shown in FIG. 112. Because image 11210 isgenerated with light sources that face away from the user of near-to-eyedisplay device 11200, the user does not see image 11210.

Various embodiments of pupil tracker units are now described. FIG. 113shows a perspective view of a near-to-eye display device that includes aLED array. Near-to-eye display device 1130 includes SLM 11320, infrared(IR) camera and light source 11302, and LED array 11310. Near-to-eyedisplay device 11300 may also include additional components, such as anelectronics module, battery, cabling, and the like. These additionalcomponents are intentionally omitted from the figure so as to notobscure the components that are shown. Further, near-to-eye displaydevice 11300, like many other near-to-eye display devices depictedherein, shows most components for only one side (one eye) of the device.In some embodiments, all components are duplicated and mirrored tocreate a near-to-eye display device for both eyes.

In some embodiments, the IR light sources are used to illuminate auser's pupils and the cameras are used to detect the position of theuser's pupils. In some embodiments, the cameras are positioned on theframe as shown in FIG. 113, although this is not a limitation of thepresent invention. For example, in some embodiments cameras are mountedon a back-light unit or are coupled into an optical path as describedabove. Cameras for pupil tracking may be placed anywhere on anynear-to-eye display device described herein without departing from thescope of the present invention. Further, in some embodiments, the IRlight sources are co-located with the cameras, although this is not alimitation of the present invention. For example, in some embodiments,IR light sources are co-located with point light sources used toilluminate an SLM. As a further example, an IR light source may beco-located with LED array 11310.

In operation, the user's eyes are illuminated with infrared light, whichis not visible to the user. The cameras capture infrared images of theuser's eyes, and existing computer vision, pattern recognition, andimage processing algorithms are used to detect the pupil positions.

FIG. 114 shows a two-dimensional LED array. LED array 11310 includes atwo-dimensional array of color light sources, where each light sourceincludes a red, a green, and a blue LED. LED array 11310 also includesLED drivers 11410. When different LEDs are selected to provide light toilluminate SLM 11320, the resulting virtual-scene wave moves slightly onthe exit pupil plane. As described below, this phenomenon is exploitedto steer the useful portion of the exit pupil plane to follow eyemotion.

FIGS. 115 and 116 show a top view of pupil tracking using multiple LEDs.The views in FIGS. 115 and 116 depict the salient components from FIG.113, and are not necessarily to scale. Further, FIGS. 115 and 116 show aone-dimensional array of three LEDs for simplicity, however in apractical system many more LEDs may be used, and a two-dimensional arraysuch as that shown in FIG. 114 may be used.

SLM 11320 is a stationary SLM that includes a reflector 11510 to reflectmodulated light as a converging beam. In some embodiments, SLM 11320 isa transmissive SLM in a converging or diverging light path. Further, insome embodiments, SLM 11320 is a reflective SLM in a converging ordiverging light path. For example, SLM 11320 may be oriented as shown inany of FIGS. 17-28.

FIG. 115 represents the case in which the user is looking straightahead, and the center LED is turned and used as the point light sourceto illuminate the SLM. FIG. 116 represents the case in which the userhas moved her eye to look a few degrees to the right. Pupil tracker11610 detects the new pupil position and commands LED driver 11410 touse a different LED to illuminate the SLM so that the useful portion ofthe exit pupil plane follows the user's pupil.

Pupil tracker 11610 may include light sources, cameras, a processor,instructions stored in a memory, and many other components. In someembodiments, pupil tracker 1160 is a combination of components, thatwhen taken together, functions to track the position of the user'spupil. As the user's pupil is tracked, pupil tracker 11610 takes one ormore actions to steer the useful portion of the exit pupil plane tofollow the user's pupil. In the case of near-to-eye display device11300, pupil tracker 11610 commands different LEDs to illuminate the SLMto steer the useful portion of the exit pupil plane to track the user'spupils.

FIG. 117 shows a near-to-eye display device that includes a rotatingSLM. Near-to-eye display device 11700 includes rotating SLM 11720,actuator 11710, camera 11302, and point light source 120. Actuator11710, when actuated, causes SLM 11720 to rotate. In some embodiments,actuator 11710 may be a stepper motor or a like device capable ofcontrolling the amount of rotation of the SLM. In some embodiments,actuator 11710 is commanded to operate by an electronic module (notshown) that is part of a pupil tracker such as pupil tracker 11610.

FIGS. 118 and 119 show a top view of pupil tracking using a rotatingSLM. The views in FIGS. 118 and 119 depict the salient components fromFIG. 117, and are not necessarily to scale. Rotating SLM 11720 includesa reflector 11510 to reflect modulated light as a converging beam. Insome embodiments, SLM 11720 is a transmissive SLM in a converging ordiverging light path. Further, in some embodiments, SLM 11720 is areflective SLM in a converging or diverging light path. For example, SLM11720 may be oriented as shown in any of FIGS. 17-28.

FIG. 118 represents the case in which the user is looking straightahead, and the rotating SLM 11720 is oriented so that the useful portionof the exit pupil plane overlaps the user's pupil. FIG. 119 representsthe case in which the user has moved her eye to look a few degrees tothe left. Pupil tracker 11610 detects the new pupil position andcommands actuator 11710 to rotate SLM 11720 so that the useful portionof the exit pupil plane follows the user's pupil.

As discussed above pupil tracker 11610 may take many forms, and manytake any appropriate action to ensure that the useful portion of theexit pupil plane tracks the user's pupil. In the case of near-to-eyedisplay device 11700, pupil tracker 11610 commands an actuator to rotatethe SLM to steer the useful portion of the exit pupil plane to track theuser's pupils.

FIG. 120 shows a perspective view of a near-to-eye display device thatincludes rotating SLMs and LED arrays. Near-to-eye display device 12000includes an LED array 11310 and rotating SLM 11720 with actuator 11710.Near-to-eye display device 12000 may rotate the SLM and select differentLEDs in any combination to steer the useful portion of the exit pupilplane to the location of the user's pupil. One example is provided inFIG. 121.

FIGS. 121 and 122 show flowcharts of methods in accordance with variousembodiments of the invention. In some embodiments, the methods of FIGS.121 and 122, or portions thereof, are performed by a near-to-eye displaydevice, embodiments of which are shown in previous figures. In otherembodiments, the methods are performed by a computer or an electronicsystem. The various actions in the methods may be performed in the orderpresented, or may be performed in a different order. Further, in someembodiments, some actions listed in FIGS. 121 and 122 are omitted.

FIG. 121 shows a flowchart showing rotation for small angles and LEDselection for larger angles. At 12110, a user's pupil is tracked. Insome embodiments, this corresponds to pupil tracker 11610 tracking theposition of a user's pupil.

When a user moves her eye, the eye rotates and the pupil moves throughan angle. When the pupil moves through a small angle, a rotatable SLM isrotated to steer the useful portion of the exit pupil plane to thelocation of the user's pupil at 12120. For larger angles, a differentlight source is selected to steer the useful portion of the exit pupilplane to the location of the user's pupil at 12130. This process isrepeated as the user moves her eye and it is tracked by the near-to-eyedisplay device.

FIG. 122 shows a flowchart showing rotation for small angles anddiffraction order selection for larger angles. At 12110, a user's pupilis tracked. In some embodiments, this corresponds to pupil tracker 11610tracking the position of a user's pupil.

When the pupil moves through a small angle, a rotatable SLM is rotatedto steer the useful portion of the exit pupil plane to the location ofthe user's pupil at 12120. For larger angles, the light wavedistribution is recomputed such that a higher diffraction order movesinto the useful portion of the exit pupil plane at 12230. This processis repeated as the user moves her eye and it is tracked by thenear-to-eye display device.

At 12110, a user's pupil is tracked. In some embodiments, thiscorresponds to pupil tracker 11610 tracking the position of a user'spupil.

When a user moves her eye, the eye rotates and the pupil moves throughan angle. When the pupil moves through a small angle, a rotatable SLM isrotated to steer the useful portion of the exit pupil plane to thelocation of the user's pupil at 12120. For larger angles, a differentlight source is selected to steer the useful portion of the exit pupilplane to the location of the user's pupil at 12130. This process isrepeated as the user moves her eye and it is tracked by the near-to-eyedisplay device.

FIG. 123 shows a near-to-eye display device that includes an activegrating. Near-to-eye display device 12300 includes SLM with activegrating 12320, actuator 12310, camera 11302, and point light source 120.Actuator 12310, when actuated, causes an active grating within SLM 12320to change its optical qualities. In some embodiments, actuator 12310 maybe a driver circuit capable of controlling a voltage applied to theactive grating. In some embodiments, actuator 12310 is commanded tooperate by an electronic module (not shown) that is part of a pupiltracker such as pupil tracker 11610.

FIGS. 124 and 125 show a top view of pupil tracking using an SLM with anactive grating. The views in FIGS. 124 and 125 depict the salientcomponents from FIG. 123, and are not necessarily to scale. SLM withactive grating 12320 includes active grating 12410, and a reflector11510 to reflect modulated light as a converging beam. In someembodiments, active grating 12410 is a custom liquid crystal baseddevice that implements a multi-section prism. Active grating 12410 maybe an LC device that merely contain electrodes and no pixels.

In some embodiments, the SLM, active grating, and reflector are separatedevices. In these embodiments, SLM 12320 may be a transmissive SLM in aconverging or diverging light path. Further, in some embodiments, SLM12320 is a reflective SLM in a converging or diverging light path. Forexample, SLM 12320 may be oriented as shown in any of FIGS. 17-28.

FIG. 124 represents the case in which the user is looking straightahead, and active grating 12410 is controlled so that the useful portionof the exit pupil plane overlaps the user's pupil. FIG. 125 representsthe case in which the user has moved her eye to look a few degrees tothe right. Pupil tracker 11610 detects the new pupil position andcommands actuator 12310 to energize active grating 12410 so that theuseful portion of the exit pupil plane follows the user's pupil.

As discussed above pupil tracker 11610 may take many forms, and manytake any appropriate action to ensure that the useful portion of theexit pupil plane tracks the user's pupil. In the case of near-to-eyedisplay device 12300, pupil tracker 11610 commands an actuator toenergize an active grating to steer the useful portion of the exit pupilplane to track the user's pupils.

FIG. 126 shows a perspective view of a near-to-eye display device thatincludes an SLM with an active grating and LED arrays. Near-to-eyedisplay device 12600 includes an LED array 11310 and SLM with activegrating 12320 with actuator 12310. Near-to-eye display device 12600 mayenergize the active grating and select different LEDs in any combinationto steer the useful portion of the exit pupil plane to the location ofthe user's pupil. One example is provided in FIG. 127.

FIGS. 127 and 128 show flowcharts of methods in accordance with variousembodiments of the invention. In some embodiments, the methods of FIGS.127 and 128, or portions thereof, are performed by a near-to-eye displaydevice, embodiments of which are shown in previous figures. In otherembodiments, the methods are performed by a computer or an electronicsystem. The various actions in the methods may be performed in the orderpresented, or may be performed in a different order. Further, in someembodiments, some actions listed in FIGS. 127 and 128 are omitted.

FIG. 127 shows a flowchart showing grating actuation for small anglesand LED selection for larger angles. At 12110, a user's pupil istracked. In some embodiments, this corresponds to pupil tracker 11610tracking the position of a user's pupil.

When a user moves her eye, the eye rotates and the pupil moves throughan angle. When the pupil moves through a small angle, an active gratingis actuated to steer the useful portion of the exit pupil plane to thelocation of the user's pupil at 12720. For larger angles, a differentlight source is selected to steer the useful portion of the exit pupilplane to the location of the user's pupil at 12730. This process isrepeated as the user moves her eye and it is tracked by the near-to-eyedisplay device.

FIG. 128 shows a flowchart showing grating actuation for small anglesand diffraction order selection for larger angles. At 12110, a user'spupil is tracked. In some embodiments, this corresponds to pupil tracker11610 tracking the position of a user's pupil.

When the pupil moves through a small angle, an active grating isenergized to steer the useful portion of the exit pupil plane to thelocation of the user's pupil at 12720. For larger angles, the light wavedistribution is recomputed such that a higher diffraction order movesinto the useful portion of the exit pupil plane at 12830. This processis repeated as the user moves her eye and it is tracked by thenear-to-eye display device.

FIGS. 129 and 130 show augmented-reality views demonstrating a virtualscene at different depths. The views in FIGS. 129 and 130 represent whata user of a near-to-eye display device might see out of one eye atdifferent accommodations. Referring now to FIG. 129, the real-world viewincludes objects in a foreground 12820, and objects in a background12940. In the example of FIG. 129, the user's accommodation is set tofocus on the foreground, hence the real-world foreground 12920 is shownin focus, and the real-world background 12940 is shown slightly out offocus.

FIG. 129 also shows a virtual scene that is superimposed on thereal-world view. In the example of FIG. 129, the virtual scene includesthree objects: virtual object 12910, virtual object 12930, and virtualobject 12950. These virtual objects are simply text, however virtualobjects can be anything, and are not limited to text. When the virtualscene was computed (see FIGS. 56-64), virtual object 12910 was set at adepth corresponding to the depth of the real-world foreground, andvirtual object 12930 was set at a depth corresponding to the depth ofthe real-world background. Further, both virtual objects 12920 and 12930are reconstructed over the entire useful portion of the exit pupilplane. This results in virtual objects 12910 and 12930 appearing focusedon the user's retina only when the user accommodates to the depth of thevirtual object. In the example of FIG. 129, the user has accommodated tothe depth of the real-world foreground, and so virtual object 12910 isalso in focus.

FIG. 130 shows the same real-world view and superimposed virtual sceneas FIG. 129. The only difference is now the user has accommodated to thedepth of the real-world background. As a result, both the real-worldbackground 12940 and the virtual object 12930 are in focus, and both thereal-world foreground 12920 and the virtual object 12910 are not infocus.

Note that virtual object 12950 is always in focus regardless of theuser's accommodation. This is because virtual object 1250 isreconstructed over a smaller region of the useful portion of the exitpupil plane, thereby increasing the depth of field. For example, in someembodiments, the virtual scene is computed in such a way that virtualobject 12950 only overlaps a one mm section of the pupil.

FIGS. 129 and 130 are an example of an SLM being programmed to displayvirtual objects appearing at different depths while some objects appearfocused in all depths (stay in focus even if the eye accommodates to adifferent depth). Waves from a first plurality of subsections of thedisplayed virtual scene are reconstructed over the entire useful portionso that each of the first plurality of subsections appears focused onthe retina only when the user accommodates to the depth of thatsubsection, and waves from a second subsection of the displayed virtualscene are reconstructed over smaller regions of the useful portion sothat these parts always appear focused on the retina.

In some embodiments, the techniques demonstrated in FIGS. 129 and 130are combined with binocular disparity to provide realistic 3D visualexperiences without causing visual fatigue due to theaccommodation-convergence conflict. When viewing 3D images usingnear-to-eye display devices described herein, eyes converge to theapparent position of a virtual 3D object and accommodation of each eyeis also set for the depth corresponding to the apparent position of thevirtual 3D object. This results in “natural 3D” in which theaccommodation-convergence conflict is greatly reduced if not completelyeliminated, providing a very comfortable 3D experience for the user.

FIG. 131 shows a block diagram of a near-to-eye display device inaccordance with various embodiments of the present invention.Near-to-eye display device 13100 includes processor 13102, memory 13110,light sources 13160, SLMs 13162, light bars 13164, cameras 13166,actuators 13168, transducers 13170, global positioning system (GPS)receiver 13172, accelerometers 13174, compass 13176, radios 13178,graphics processing unit (GPU) 13180, gyroscopes 13182, touchscreen13184, and audio circuits 13186. Near-to-eye display device 13100 may beany near-to-eye display device described herein. For example, in someembodiments, mobile device 300 may be a near-to-eye display device thatperforms pupil filtering, pupil tracking, speckle reduction, or anyother function described herein.

Processor 13102 may be any type of processor capable of executinginstructions store in memory 13110 and capable of interfacing with thevarious components shown in FIG. 131. For example, processor 13102 maybe a microprocessor, a digital signal processor, an application-specificprocessor, or the like. In some embodiments, processor 13102 is acomponent within a larger integrated circuit such as a system on chip(SOC) application-specific integrated circuit (ASIC).

Light sources 13160 may include any type of light source capable ofilluminating an SLM. Examples include point light source 120 (FIG. 1),illumination optics module 440 (FIG. 4), and the array of point lightsources shown in FIGS. 15 and 16. In operation, processor 13102 maycommand light sources 13160 to turn on and off.

SLMs 13162 are SLMs that impart information to an illumination wave tocreate the desired light wave distribution in the useful portion of theexit pupil plane. In operation, processor 13102 programs SLMs 13162using data stored in memory 13110. In some embodiments, processor 13102computes the SLM data to be displayed on the SLM and stores it in memory13110. In other embodiments, the SLM data is computed by a separatedevice, and the SLM data is provided to near-to-eye display device 13100for later display.

Light bars 13164 include any of the light bar and/or moving platformembodiments described herein. In operation, processor 13102 may commandan actuator to cause one or more light bar to move. Further processor13102 may also command one or more light sources on a light bar toilluminate.

Cameras 13166 may be any type of camera capable of capturing an imageand providing the image data to processor 13102. For example, in someembodiments, cameras 13166 are cameras used for calibration, and inother embodiments, cameras 13166 are cameras used for pupil tracking.

Actuators 13168 are devices that convert one form of energy to another.For example, actuators 13168 may include stepper motors, magnets,electrical coils, and the like. Actuators 13168 may include any of theactuator embodiments described herein.

Transducers 13170 are devices that convert energy from one form toelectricity. For example, adjustment knob 4510 (FIG. 45) is an exampleof a transducer. In operation, processor 13102 receives electronicsignals when a user interacts with any of transducers 13170.

GPS 13172 includes a GPS receiver. In operation, processor 13102receives fine location data from GPS 13172. In some embodiments, thisdata is used to generate SLM data or to determine what stored SLM datashould be displayed. For example, in embodiments represented FIGS. 120and 130, GPS data may be used to determine what virtual objects shouldbe included in the virtual scene.

Accelerometers 13174 are devices that measure rate of change of motionor the direction of forces applied to near-to-eye display device 13100due to gravity. In operation, processor 13102 receives accelerometerdata when near-to-eye display device 13100 is moved or its orientationis changed.

Compass 13176 is a device that measures the orientation of near-to-eyedisplay device 13100 relative to magnetic north. In operation, processor13102 receives data from compass 13176 that represents the orientationof near-to-eye display device 13100 with respect to magnetic north.

Radios 13178 may include any type of radio that can providecommunications capability to near-to-eye display device 13100. Forexample, radio 13178 may be a cellular radio, a Bluetooth radio, a NFCradio, a WiFi radio, or the like.

Graphics processing unit (GPU) 13180 is a device that can acceleratesome computations performed during the generation of SLM data. Forexample, GPU 13180 may be used to render a virtual scene represented bypolygon mesh models.

Gyroscopes 13182 provide high-resolution data regarding movement ofnear-to-eye display device. In operation, processor 13102 may make useof data provided by gyroscopes 13182 for head-tracking applications.

Touchscreen 13184 allows user interaction with the display surfaces ofnear-to-eye display device 13100. An example near-to-eye display devicewith a touchscreen interface is described below with reference to FIG.132. Touchscreen 13184 is a device that includes a touch-sensitivesurface, sensor, or set of sensors that accept input from a user. Forexample, touchscreen 13184 may detect when and where an object touchesthe screen, and may also detect movement of an object across the screen.Touchscreen 13184 may be manufactured using any applicable displaytechnologies, including for example, liquid crystal display (LCD),active matrix organic light emitting diode (AMOLED), and the like.Further, touchscreen 13184 may be manufactured using any applicationtouch-sensitive input technologies, including for example, capacitiveand resistive touch screen technologies, as well as other proximitysensor technologies.

Audio circuits 13186 provide an audio interface (input, output, or both)between processor 13102 and a user. In some embodiments, one or moreapplications make use of audio circuits 13186 to provide a multi-sensoryexperience. For example, tour-guide application 13143 may provideinterpretive audio as well as an immersive 3D augmented-realityexperience. In other embodiments, audio circuits 13186 include amicrophone that allows a user to record audio or to provide audiocommands to near-to-eye display device 13100.

Memory 13110 may include any type of memory device. For example, memory13110 may include volatile memory such as static random-access memory(SRAM), or nonvolatile memory such as FLASH memory. Memory 13110 isencoded with (or has stored therein) one or more software modules (orsets of instructions), that when accessed by processor 113102, result inprocessor 13102 performing various functions. In some embodiments, thesoftware modules stored in memory 13110 may include an operating system(OS) 13120, near-to-eye modules 13130 and applications 13140.Applications 13140 may include any number or type of applications.Examples provided in FIG. 131 include games 13141, maps 13142, atour-guide app 13143, and a video player. An example display from atour-guide app is described above with reference to FIGS. 129 and 130.Memory 13110 may also include any amount of space dedicated to datastorage 13150.

Operating system 13120 may be any to of operating system such as anoperating system to control a mobile phone, tablet computer, embeddedsystem, or the like. As shown in FIG. 131, operating system 13120includes user interface component 13121 and application-installercomponent 13122. Operating system 13120 may include many othercomponents without departing from the scope of the present invention.

User interface component 13121 includes processor instructions thatcause near-to-eye display device 13100 to display user interactioncomponents, such as dialog boxes, alerts, and prompts. User interface13121 also includes instructions to display menus, move icons, andmanage other portions of the display environment.

Application-installer component 13122 installs applications tonear-to-eye display device 13100. Any type or number of applications maybe installed. Example apps currently installed on near-to-eye displaydevice include games 13141, maps 13142, tour-guide app 13143, andvideo-player app 13144.

Near-to-eye modules 13130 include calibration 13131, SLM computation13132, pupil tracking 13133, and speckle reduction 13134. Calibrationmodule 13131 includes instructions that cause processor 13102 to performcalibration embodiments described herein. For example, calibrationmodule 13131 may cause processor 13102 to capture images using cameras13166, and interact with the user using user interface 13121 andtransducers 13170. SLM computation module includes instructions toperform the computations described above with reference to FIG. 56. Thenear-to-eye modules shown in FIG. 131 are meant as examples only; manymore near-to-eye modules may be included without departing from thescope of the present invention. In general, any method described hereinmay include a module component within near-to-eye modules 13130.

Pupil-tracking module 13133 includes instructions that when executed byprocessor 13102 cause near-to-eye display device 13100 to steer theuseful portion of the exit pupil plane to follow a user's pupils. Insome embodiments, the combination of pupil-tracking module 13133,processor 13102, cameras 13166, and light sources 13160 (for IR light)make up pupil tracker 11610 described above.

Speckle reduction module 13134 includes instruction that when executedby processor 13102 causes a virtual scene to be computed with assignedphase terms that reduce speckle.

Data storage 13150 stores data that does not include processorinstructions. For example, SLM data 13151 is stored in data storage13150, as are user profiles. In some embodiments, SLM data 13151includes still images, and in other embodiments, SLM data 13151 includesmany frames that form video data. Further, SLM data 13151 may represent2D or 3D virtual scenes used for either or both ofvirtual-reality-display applications or augmented-reality applications.

Each of the above-identified applications and modules correspond to aset of instructions for performing one or more functions describedabove. These applications (sets of instructions) need not be implementedas separate software programs, procedures or modules, and thus varioussubsets of these applications may be combined or otherwise re-arrangedin various embodiments. For example, SLM computation 13132 may becombined with speckle reduction 13134. Furthermore, memory 13110 maystore additional applications (e.g., audio players, camera applications,etc.) and data structures not described above.

It should be noted that device 13100 is presented as an example of anear-to-eye display device, and that device 13100 may have more or fewercomponents than shown, may combine two or more components, or may have adifferent configuration or arrangement of components. For example,device 13100 may include many more components such as sensors (optical,touch, proximity etc.), or any other components suitable for use in anear-to-eye display device.

Memory 13110 represents a computer-readable medium capable of storinginstructions, that when accessed by processor 13102, result in theprocessor performing as described herein. For example, when processor13102 accesses instructions within pupil-tracking module 13133,processor 13102 analyzes images of a user's eyes, determines the pupillocation, and then steers the useful portion of the exit pupil plan tooverlap with the user's pupil.

FIG. 132 shows a near-to-eye display device with transparenttouch-sensitive layers 13210. In some embodiments, the front surfaces ofthe near-to-eye display device are covered with transparenttouch-sensitive layers that allow for user interaction. For example, auser using near-to-eye display device 13200 can use her fingers to makeselections among displayed items 13220 (e.g., some icons/menu items) orto perform actions such as zoom-in and -out operations, and input textdata through virtual keyboards, similar to the usage of touch-sensitivescreens on existing smart phones, tablets, etc., with the differencethat the user sees the displayed content through the backside of thedisplay, while she performs the finger touch based input operationsthrough the front side.

The following paragraphs provide further disclosure of various inventionembodiments. Each embodiment is fully defined by the recitation of thecorresponding paragraph, and no other elements are to be consideredessential for that particular embodiment. The embodiments include:

1A1. A near-to-eye display device comprising:

at least one point light source; and

at least one spatial light modulator (SLM) mounted on the near-to-eyedisplay device;

wherein light produced by the at least one point light sourceilluminates the SLM and gets modulated to produce modulated light, andthe modulated light is directed on an exit pupil plane that includes auseful portion, and wherein a light wave distribution within the usefulportion is equal to a computed light distribution from a virtual scene;

and wherein the useful portion is steerable across the exit pupil planeto follow the motion of a user's eye pupil when the near-to-eye displaydevice is in use so that the user's eye pupil acts as a spatial filterto filter out undesired beams produced by the SLM at the exit pupilplane.

1A2. A near-to-eye display device comprising:

at least one point light source; and

at least one spatial light modulator (SLM) mounted on the near-to-eyedisplay device;

wherein light produced by the at least one point light sourceilluminates the SLM and gets modulated to produce modulated light, andthe modulated light is directed on an exit pupil plane that includes auseful portion, and wherein a light wave distribution within the usefulportion is equal to a computed light distribution from a virtual scene;

and wherein the useful portion is steerable to an expected location of auser's eye pupil when the near-to-eye display device is in use so thatthe user's eye pupil acts as a spatial filter to filter out undesiredbeams produced by the SLM at the exit pupil plane.

1A3. A near-to-eye display device comprising:

at least one point light source; and

at least one spatial light modulator (SLM) mounted on the near-to-eyedisplay device;

wherein light produced by the at least one point light sourceilluminates the SLM and gets modulated to produce modulated light, andthe modulated light is directed on an exit pupil plane that includes auseful portion, and wherein a light wave distribution within the usefulportion is equal to a computed light distribution from a virtual scene;

and wherein the light wave distribution is determined using acomputation that adds a controlled phase variation on the virtual sceneto reduce speckle.

1A4. A near-to-eye display device comprising:

at least one point light source; and

at least one spatial light modulator (SLM) mounted on the near-to-eyedisplay device;

wherein light produced by the at least one point light sourceilluminates the SLM and gets modulated to produce modulated light, andthe modulated light is directed on an exit pupil plane that includes auseful portion, and wherein a light wave distribution within the usefulportion is equal to a computed light distribution from a virtual scene;

and wherein the light wave distribution is determined using acomputation that adds a phase delay variation to the virtual-scenepoints such that individual waves from the virtual-scene points arriveto the useful portion in-phase to reduce speckle.

1A5. A near-to-eye display device comprising:

at least one point light source; and

at least one spatial light modulator (SLM) mounted on the near-to-eyedisplay device;

wherein light produced by the at least one point light sourceilluminates the SLM and gets modulated to produce modulated light, andthe modulated light is directed on an exit pupil plane that includes auseful portion, and wherein a light wave distribution within the usefulportion is equal to a computed light distribution from a virtual scene;

and wherein the light wave distribution is determined using acomputation that adds a phase delay variation to the virtual-scenepoints such that optical path lengths between the useful portion and thevirtual-scene points differ by an integer multiple of a centerwavelength of the at least one light source.

1A6. A near-to-eye display device comprising:

at least one point light source; and

at least one spatial light modulator (SLM) mounted on the near-to-eyedisplay device;

wherein light produced by the at least one point light sourceilluminates the SLM and gets modulated to produce modulated light, andthe modulated light is directed on an exit pupil plane that includes auseful portion, and wherein a light wave distribution within the usefulportion is equal to a computed light distribution from a virtual scene;

and wherein an image viewed through the useful portion of the exit pupilplane exhibits reduced speckle generated by controlling a phase ofvirtual object points.

1A7. A near-to-eye display device comprising:

at least one point light source; and

at least one spatial light modulator (SLM) mounted on the near-to-eyedisplay device;

wherein light produced by the at least one point light sourceilluminates the SLM and gets modulated to produce modulated light, andthe modulated light is directed on an exit pupil plane that includes auseful portion, and wherein a light wave distribution within the usefulportion is equal to a computed light distribution from a virtual scene;

and wherein the light wave distribution is determined using acomputation that compensates for optical aberrations of a user's unaidedeye.

1A8. A near-to-eye display device comprising:

at least one point light source; and

at least one spatial light modulator (SLM) mounted on the near-to-eyedisplay device;

wherein light produced by the at least one point light sourceilluminates the SLM and gets modulated to produce modulated light, andthe modulated light is directed on an exit pupil plane that includes auseful portion, and wherein a light wave distribution within the usefulportion is equal to a computed light distribution from a virtual scene;

and wherein the SLM data is determined using a computation thatcompensates for optical aberrations of a light path from the at leastone point light source to the exit pupil plane.

1A9. A near-to-eye display device comprising:

at least one point light source; and

at least one spatial light modulator (SLM) mounted on the near-to-eyedisplay device;

wherein light produced by the at least one point light sourceilluminates the SLM and gets modulated to produce modulated light, andthe modulated light is directed on an exit pupil plane that includes auseful portion, and wherein a light wave distribution within the usefulportion is equal to a computed light distribution from a virtual scene;

wherein the useful portion is steerable across the exit pupil plane tofollow the motion of a user's eye pupil when the near-to-eye displaydevice is in use;

wherein the user's eye pupil acts as a spatial filter to filter outundesired beams produced by the SLM at the exit pupil plane;

wherein the waves from a first plurality of subsections of the displayedvirtual scene are reconstructed over the entire useful portion so thateach of the first plurality of subsections appears focused on the retinaonly when the user accommodates to the depth of that subsection;

and wherein the waves from a second subsection of the displayed virtualscene are reconstructed over smaller regions of the useful portion sothat these parts always appear focused on the retina.

1A10. A device in accordance with any of paragraphs 1A1-1A34, whereinthe SLM produces higher diffraction orders that fall outside the usefulportion.

1A11. A device in accordance with any of paragraphs 1A1-1A34, whereinthe SLM produces quantization noise that falls outside the usefulportion.

1A12. A device in accordance with any of paragraphs 1A1-1A34, whereinthe SLM produces conjugate beams that fall outside the useful portion.

1A13. A device in accordance with any of paragraphs 1A1-1A34, whereinthe SLM produces a DC beam that falls outside the useful portion.

1A14. A device in accordance with any of paragraphs 1A1-1A34, whereinthe virtual scene is two-dimensional.

1A15. A device in accordance with any of paragraphs 1A1-1A34, whereinthe virtual scene is three-dimensional.

1A16. A device in accordance with any of paragraphs 1A1-1A34, whereinthe modulated light is focused onto the exit pupil plane.

1A17. A device in accordance with any of paragraphs 1A1-1A34, whereinthe at least one light source comprising a plurality of light sourcesthat produce light of different wavelengths.

1A18. A device in accordance with any of paragraphs 1A1-1A34, whereinthe at least one light source comprising a red light source, a greenlight source, and a blue light source.

1A19. A device in accordance with any of paragraphs 1A1-1A34, whereinthe useful portion of the exit pupil plane substantially overlaps with auser's eye pupil when the near-to-eye display device is in use.

1A20. A device in accordance with any of paragraphs 1A1-1A34, whereinthe useful portion of the exit pupil plane is at least as large as anexpected size of a user's eye pupil when the near-to-eye display deviceis in use.

1A21. A device in accordance with any of paragraphs 1A1-1A34, wherein auseful portion of the exit pupil plane matches an expected size of theuser's eye pupil.

1A22. A device in accordance with any of paragraphs 1A1-1A34, whereinthe light illuminating the spatial light modulator converges, and theuseful portion of the exit pupil plane includes a single diffractionorder.

1A23. A device in accordance with any of paragraphs 1A1-1A34, whereinthe near-to-eye display device comprises a head-worn device.

1A24. A device in accordance with any of paragraphs 1A1-1A34, wherein aratio of an optical distance between the spatial light modulator and theexit pupil plane to the pixel pitch is greater than an expected pupilsize divided by a smallest wavelength of light emitted by the at leastone point light source.

1A25. A device in accordance with any of paragraphs 1A1-1A34, whereinthe spatial light modulator is in a light path between the at least onepoint light source and a pupil and not in an optical conjugate plane toa user's retina when the near-to-eye display device is in use.

1A26. A device in accordance with any of paragraphs 1A1-1A34, whereinlight projected onto the exit pupil plane includes multiple diffractionorders produced by the spatial light modulator, and the useful portionincludes a single diffraction order.

1A27. A device in accordance with any of paragraphs 1A1-1A34, wherein awidth of the useful portion is greater than an expected width of auser's eye pupil.

1A28. A device in accordance with any of paragraphs 1A1-1A34, wherein awidth of the useful portion is greater than 3 mm.

1A29. A device in accordance with any of paragraphs 1A1-1A34, whereinthe light projected on the exit pupil plane includes multiple imagecopies, and the useful portion includes one image copy.

1A30. A device in accordance with any of paragraphs 1A1-1A34, whereinthe spatial light modulator modulates only phase of the lightilluminating the SLM.

1A31. A device in accordance with any of paragraphs 1A1-1A34, whereinthe spatial light modulator modulates only amplitude of the lightilluminating the SLM.

1A32. A device in accordance with any of paragraphs 1A1-1A34, whereinthe spatial light modulator modulates phase and amplitude.

1A33. A device in accordance with any of paragraphs 1A1-1A34, whereinthe spatial light modulator is reflective.

1A34. A device in accordance with any of paragraphs 1A1-1A34, whereinthe spatial light modulator is transmissive.

1A35. A device in accordance with any of paragraphs 1A1-1A34, whereinthe useful portion is steerable across the exit pupil plane to followthe motion of a user's eye pupil when the near-to-eye display device isin use.

1A36. A device in accordance with any of paragraphs 1A1-1A34, whereinthe useful portion is steerable to an expected location of a user's eyepupil when the near-to-eye display device is in use.

1A37. A device in accordance with any of paragraphs 1A1-1A34, whereinthe light wave distribution is determined using a computation that addsa controlled phase variation on the virtual scene to reduce speckle.

1A38. A device in accordance with any of paragraphs 1A1-1A34, whereinthe light wave distribution is determined using a computation that addsa phase delay variation to the virtual-scene points such that individualwaves from the virtual-scene points arrive to the useful portionin-phase to reduce speckle.

1A39. A device in accordance with any of paragraphs 1A1-1A34, whereinthe light wave distribution is determined using a computation that addsa phase delay variation to the virtual-scene points such that opticalpath lengths between the useful portion and the virtual-scene pointsdiffer by an integer multiple of a center wavelength of the at least onelight source.

1A40. A device in accordance with any of paragraphs 1A1-1A34, wherein animage viewed through the useful portion of the exit pupil plane exhibitsreduced speckle generated by controlling a phase of virtual objectpoints.

1A41. A device in accordance with any of paragraphs 1A1-1A34, whereinthe light wave distribution is determined using a computation thatcompensates for optical aberrations of a user's unaided eye.

1A42. A device in accordance with any of paragraphs 1A1-1A34, whereinthe SLM data is determined using a computation that compensates foroptical aberrations of a light path from the at least one point lightsource to the exit pupil plane.

1B1. A near-to-eye display device comprising:

an array of point light sources mounted to the near-to-eye displaydevice; and

a spatial light modulator illuminated by the array of point lightsources in a time sequential manner, the spatial light modulator havinga plurality of sections that project diverging light toward an exitpupil plane positioned at an expected location of a user's eye pupilwhen the near-to-eye display device is in use;

wherein the spatial light modulator and the array of point light sourcesare positioned such that each of the plurality of sections contributesto the light wave in the useful portion of the exit pupil plane with thehighest optical power when the corresponding point light source of thearray is turned on.

1B2. The near-to-eye display device of 1B1 wherein the array of pointlight sources comprises a plurality of groups of point light sources,with more than one point light source in a group, and the point lightsources within each of the plurality of groups can be turned on at thesame time.

1B3. The near-to-eye display device of 1B1, wherein the near-to-eyedisplay device comprises a head-worn device.

1B4. A near-to-eye display device comprising:

an array of point light sources with restricted emission cones mountedto the near-to-eye display device; and

a spatial light modulator illuminated simultaneously by the array ofpoint light sources with restricted emission cones, the spatial lightmodulator having a plurality of sections that project diverging lighttoward an exit pupil plane positioned at an expected location of auser's eye pupil when the near-to-eye display device is in use;

wherein the spatial light modulator and the array of point light sourcesare positioned such that each of the plurality of sections areilluminated only by one of the point light sources in the array.

1B5. The near-to-eye display device of 1B4 further comprising a secondarray of point light sources with restricted emission cones, wherein thearray of point light sources and the second array of point light sourcespartition the SLM differently with nonoverlapping borders, and whereinthe array of point light sources and the second array of point lightsources are turned on in a time sequential manner.

1B6. The near-to-eye display device of 1B4, wherein the near-to-eyedisplay device comprises a head-worn device.

1B7. A method comprising:

determining a plurality of data sets to be programmed in a spatial lightmodulator (SLM) in a near-to-eye display device that includes an arrayof point light sources, wherein for a video frame of a virtual scene, adifferent data set for each of the point light sources in the array iscomputed; and

displaying the plurality of data sets on the SLM in a time sequentialmanner in synchronism with a corresponding point light source within anoverall time allocated for the video frame.

1B8. A method comprising:

determining a plurality of data sets to be programmed in a spatial lightmodulator (SLM) in a near-to-eye display device that includes an arrayof point light sources with restricted emission cones, wherein eachpoint light source in the array illuminates a different section of theSLM, and wherein for a video frame of a virtual scene, one data set foreach different section of the SLM is computed according to the pointlight source which illuminates that section of the SLM; and

concatenating the plurality of data sets for the different sections toobtain a final SLM data for the video frame.

1C1. A near-to-eye display device comprising:

a point light source;

a spatial light modulator (SLM), wherein light produced by the pointlight source illuminates the SLM and gets modulated to produce modulatedlight, and the modulated light is directed on an exit pupil plane thatincludes a useful portion, and wherein a light wave distribution withinthe useful portion is equal to a computed light distribution from avirtual scene; and

a microdisplay positioned on the near-to-eye display device to generateon a user's retina a defocused peripheral image that surrounds a focusedimage generated by the spatial light modulator.

1C2. A near-to-eye display device comprising:

a point light source;

a spatial light modulator (SLM), wherein light produced by the pointlight source illuminates the SLM and gets modulated to produce modulatedlight, and the modulated light is directed on an exit pupil plane thatincludes a useful portion, and wherein a light wave distribution withinthe useful portion is equal to a computed light distribution from avirtual scene; and

a microdisplay positioned on the near-to-eye display device to generatea defocused low-resolution image that surrounds an image generated bythe spatial light modulator.

1C3. The near-to-eye display device of any of 1C1-1C2, wherein the SLMhas a first resolution and the microdisplay has a second resolution thatis lower than the first resolution.

1C4. The near-to-eye display device of any of 1C1-1C2, wherein the SLMhas a first pixel pitch and the microdisplay has a second pixel pitchthat is greater than the first pixel pitch.

1C5. The near-to-eye display device of any of 1C1-1C2, wherein themodulated light is steerable across the exit pupil plane to follow themotion of a user's eye pupil when the near-to-eye display device is inuse.

1C6. The near-to-eye display device of any of 1C1-1C2 wherein the pointlight source comprises a plurality of light sources to emit light ofdifferent wavelengths.

1C7. The near-to-eye display device of any of 1C6 wherein the pluralityof light sources emit light sequentially.

1C8. The near-to-eye display device of any of 1C1-1C2 wherein the SLM ismounted on a movable platform.

IC9. The near-to-eye display device of any of 1C1-1C2 wherein the SLM ismounted on a slotted movable platform.

IC10. The near-to-eye display device of any of 1C1-1C2 wherein the SLMincludes at least one row of pixels.

IC11. The near-to-eye display device of any of 1C1-1C2 wherein the SLMhas a vertical dimension of at least 2 mm.

IC12. The near-to-eye display device of any of 1C1-1C2 wherein the SLMpresents a horizontal field of view of about 30 degrees.

IC13. The near-to-eye display device of any of 1C1-1C2 wherein the SLMpresents a horizontal field of view of about 40 degrees.

1C14. The near-to-eye display device of any of 1C1-1C2 wherein themicrodisplay is mounted on a movable platform.

IC15. The near-to-eye display device of any of 1C1-1C2 wherein themicrodisplay is mounted on a slotted movable platform.

1C16. The near-to-eye display device of any of 1C1-1C2 wherein themicrodisplay can be selected from an organic light emitting diode (OLED)display, a transmissive liquid crystal display (LCD), or a reflectiveLCD.

1C17. The near-to-eye display device of any of 1C1-1C2, wherein thenear-to-eye display device comprises a head-worn device.

1C18. A near-to-eye display device comprising:

a spatial light modulator capable of modulating reflected light ordisplaying color pixels; and

a pupil-tracking device to track a user's pupil position; and

a spatial light modulator driver circuit responsive to thepupil-tracking device to cause the spatial light modulator to modulatereflected light in a central region of the user's gaze and to displaycolor pixels away from the central region of the user's gaze.

1C19. The near-to-eye display device of 1C18, wherein the near-to-eyedisplay device comprises a head-worn device.

1D1. In a near-to-eye display device that includes a spatial lightmodulator (SLM) to modulate incident light and direct modulated light onan exit pupil plane that includes a useful portion, wherein a light wavedistribution within the useful portion is equal to a computed lightdistribution from a virtual scene, a method comprising:

modifying the light wave distribution to present a user with a pluralityof test images intended to determine a type of visual disorder sufferedby the user;

receiving feedback from the user regarding the plurality of test images;

determining the type of visual disorder suffered by the user; and

modifying the light wave distribution to present the user with a secondplurality of test images to determine a degree of the visual disordersuffered by the user.

1D2. In a near-to-eye display device that includes a spatial lightmodulator (SLM) to modulate incident light and direct modulated light onan exit pupil plane that includes a useful portion, wherein a light wavedistribution within the useful portion is equal to a computed lightdistribution from a virtual scene, a method comprising:

prompting a user to identify a type of any visual disorder of the user;

modifying the light wave distribution to present the user with at leastone test image intended to determine a degree of the visual disorder;and

receiving feedback from the user regarding the at least one test image.

1D3. In a near-to-eye display device that includes a spatial lightmodulator (SLM) to modulate incident light and direct modulated light onan exit pupil plane that includes a useful portion, wherein a light wavedistribution within the useful portion is equal to a computed lightdistribution from a virtual scene, a method comprising:

prompting a user to identify a type and degree of any visual disorder ofthe user;

modifying the light wave distribution to present the user with at leastone test image intended to correct for the visual disorder; and

receiving feedback from the user regarding the at least one test image.

1D4. The method of any of 1D1-1D3, wherein the test image comprisesmultiple test images presented sequentially.

1D5. The method of any of 1D1-1D3, wherein the test image comprisesmultiple test images presented serially.

1D6. The method of any of 1D1-1D3, wherein the feedback comprisesselection of one of the multiple test images.

1D7. The method of any of 1D1-1D3, wherein the receiving feedbackcomprises receiving information from a transducer.

1D8. The method of 1D7 wherein the transducer comprises an adjustmentknob.

1D9. The method of any of 1D1-1D3, wherein the user selects an image andthen interacts with a transducer to provide feedback.

1D10. The method of any of 1D1-1D3, wherein the feedback from the useris used to adjust for interpupil distance variations.

1D11. The method of any of 1D1-1D3, wherein the feedback from the useris used to adjust for eye relief variations.

1D12. The method of any of 1D1-1D11 further comprising providing theuser with a corrected image.

1D13. The method of any of 1D1-1D12 wherein the near-to-eye displaydevice comprises a head-worn device.

1D14. A near-to-eye display device comprising:

at least one point light source;

at least one spatial light modulator (SLM) mounted on the near-to-eyedisplay device, wherein light produced by the at least one point lightsource illuminates the SLM and gets modulated to produce modulatedlight, and the modulated light is directed on an exit pupil plane thatincludes a useful portion, and wherein a light wave distribution withinthe useful portion is equal to a computed light distribution from avirtual scene; and

a display-calibration component to modify the light wave distributionbased on user selection of test images in order to compensate for one ormore visual disorders of the user.

1D15. The near-to-eye display device of 1D14 further comprising atransducer coupled to the display-calibration component to receive userfeedback.

1D16. The near-to-eye display device of 1D15 wherein the transducercomprises an adjustment knob.

1D17. The near-to-eye display device of 1D14 wherein thedisplay-calibration component includes a processor and a memory devicehaving instructions stored thereon that when executed by the processorperform display calibration.

1D18. The near-to-eye display device of 1D14 wherein thedisplay-calibration component modifies phase values of the lightdistribution.

1D19. The near-to-eye display device of 1D14 wherein thedisplay-calibration component performs any of the actions of 1D1-1D12

1D20. The near-to-eye display device of 1D14, wherein the near-to-eyedisplay device comprises a head-worn device.

1E1. A near-to-eye display device comprising:

at least one point light source;

at least one spatial light modulator (SLM) mounted on the near-to-eyedisplay device, wherein light produced by the at least one point lightsource illuminates the SLM and gets modulated to produce modulatedlight, and the modulated light is directed on an exit pupil plane thatincludes a useful portion, and wherein a light wave distribution withinthe useful portion is equal to a computed light distribution from avirtual scene;

a camera; and

a display-calibration component to modify data patterns presented to theSLM based on images captured by the camera.

1E2. A near-to-eye display device comprising:

at least one point light source;

at least one spatial light modulator (SLM) mounted on the near-to-eyedisplay device, wherein light produced by the at least one point lightsource illuminates the SLM and gets modulated to produce modulatedlight, and the modulated light is directed on an exit pupil plane thatincludes a useful portion, and wherein a light wave distribution withinthe useful portion is equal to a computed light distribution from avirtual scene;

a camera;

at least one actuator to modify physical characteristics of thenear-to-eye display device; and

a display-calibration component coupled to the at least one actuator tomodify the physical characteristics of the near-to-eye display devicebased on images captured by the camera.

1E3. The near-to-eye display device of any of 1E1-1E2 wherein thevirtual scene includes test images to measure user physicalcharacteristics.

1E4. The near-to-eye display device of 1E3 wherein the test images aredisplayed at different depths.

1E5. The near-to-eye display device of 1E3 wherein the test images aredisplayed at different transverse positions.

1E6. The near-to-eye display device of 1E5 wherein the test images areused to determine actuator settings to compensate for variations ininterpupil distance.

1E7. The near-to-eye display device of 1E1-1E2, wherein the near-to-eyedisplay device comprises a head-worn device.

1E8. In a near-to-eye display device that includes a spatial lightmodulator (SLM) to modulate incident light and direct modulated light onan exit pupil plane that includes a useful portion, wherein a light wavedistribution within the useful portion is equal to a computed lightdistribution from a virtual scene, a method comprising:

displaying at least one test image;

recording images of a user's eyes while viewing the at least one testimage;

analyzing the recorded images for user characteristics;

interacting with at least one actuator to compensate for the usercharacteristics.

1E9. The method of 1E8 wherein recording images of a user's eyescomprises recording the images using two cameras.

1E10. The method of 1E8 wherein analyzing the recording images for usercharacteristics comprises recording a position of the user's pupils.

1E11. The method of 1E8 wherein analyzing the recording images for usercharacteristics comprises recording an interpupil distance between theuser's pupils.

1E12. The method of 1E8 wherein analyzing the recording images for usercharacteristics comprises measuring a user's eye anomalies.

1E13. The method of 1E8 wherein the near-to-eye display device comprisesa head-worn device.

1F1. A method comprising:

determining a two-dimensional complex-valued profile of a virtual-scenewave on a useful portion of an exit pupil plane;

back-propagating the two-dimensional complex-valued profile of thevirtual-scene wave on the useful portion of the exit pupil plane to aspatial light modulator plane to determine an ideal two-dimensionalcomplex-valued wave profile at an exit of the spatial light modulator;

determining a two-dimensional complex-valued profile of an illuminationwave that will illuminate the spatial light modulator;

extracting the two-dimensional complex-valued wave profile of theillumination wave from the ideal two-dimensional complex-valued waveprofile at the exit of the spatial light modulator to obtain atwo-dimensional ideal analog complex-valued spatial light modulatortransmittance;

apply prefiltering and sampling to the two-dimensional ideal analogcomplex-valued spatial light modulator transmittance to obtain atwo-dimensional ideal complex-valued discrete spatial light modulatorimage; and

encoding the two-dimensional ideal complex-valued discrete spatial lightmodulator image into a two-dimensional actual digital spatial lightmodulator image that is suitable for displaying by the spatial lightmodulator, where noise introduced by the spatial light modulator isdistributed to regions outside the useful portion.

1F2. The method of 1F1 wherein determining a two-dimensionalcomplex-valued profile of a virtual scene on a useful portion of an exitpupil plane comprises using a point cloud object model to represent avirtual object as a plurality of point light sources.

1F3. The method of 1F2 wherein determining a two-dimensionalcomplex-valued profile of a virtual scene on a useful portion of an exitpupil plane comprises adding a spherical wave term for each of theplurality of point light sources.

1F4. The method of 1F1 wherein determining a two-dimensionalcomplex-valued profile of a virtual scene on a useful portion of an exitpupil plane comprises taking an RGB image rendered by a graphicsprocessing unit (GPU) for a viewpoint along with zBuffer data providedby the GPU, and representing the object surface facing the viewpoint asa plurality of point light sources with corresponding RGB values anddepth locations.

1F5. The method of 1F1 wherein determining a two-dimensionalcomplex-valued profile of a virtual-scene wave on a useful portion of anexit pupil plane comprises:

partitioning the virtual scene into a plurality of spherical surfacesconcentric at the center of the useful portion of the exit pupil planewith different radius;

forming a matrix for each of the spherical surfaces where each elementof the matrix is associated with a specific angular location on thesphere, and each element is filled with the complex amplitude of thepoint source at that angular location on the sphere;

inverse Fourier transforming the matrix to create a result;

multiplying the result by a common diverging lens term with a focallength equal to the radius of the sphere; and

repeating the partitioning, forming, inverse Fourier transforming, andmultiplying for each of the plurality of spherical surfaces andsuperposing to find the two-dimensional complex-valued profile of thevirtual-scene wave on the useful portion of the exit pupil plane.

1F6. The method of 1F1 wherein back-propagating comprises incorporatingfree-space propagation and wave optics models of any optical componentsincluding aberrations introduced by the components between the spatiallight modulator and the useful portion of the exit pupil plane.

1F7. The method of 1F1 wherein determining a two-dimensionalcomplex-valued profile of an illumination wave comprises performingfree-space propagation and wave optics analysis for components within anillumination module including aberrations.

1F8. The method of 1F1 wherein encoding comprises encoding as aphase-only hologram.

1F9. The method of 1F1 wherein encoding comprises encoding anamplitude-only hologram.

1F10. The method of 1F1 wherein encoding comprises encoding as a binaryhologram.

1F11. For objects closer than 25 cm, the size of the useful portion istaken to be smaller than 2 mm, a typical value being 1 mm, so that asharp retinal image of these objects can also be delivered to the retinausing the pinhole imaging principle. The images have infinite depth offocus.

1G1. A method comprising:

determining a two-dimensional complex-valued profile of a virtual-scenewave on a useful portion of an exit pupil plane using a point cloudvirtual-scene model to represent a virtual scene as a plurality ofvirtual-scene points;

assigning a phase value to each of the plurality of virtual-scene pointsto reduce speckle;

back-propagating the two-dimensional complex-valued profile of thevirtual-scene wave on the useful portion of the exit pupil plane to aspatial light modulator plane to determine an ideal two-dimensionalcomplex-valued wave profile at an exit of the spatial light modulator;

determining a two-dimensional complex-valued profile of an illuminationwave that will illuminate the spatial light modulator;

extracting the two-dimensional complex-valued wave profile of theillumination wave from the ideal two-dimensional complex-valued waveprofile at the exit of the spatial light modulator to obtain atwo-dimensional ideal analog complex-valued spatial light modulatortransmittance;

apply prefiltering and sampling to the two-dimensional ideal analogcomplex-valued spatial light modulator transmittance to obtain atwo-dimensional ideal complex-valued discrete spatial light modulatorimage; and

encoding the two-dimensional ideal complex-valued discrete spatial lightmodulator image into a two-dimensional actual digital spatial lightmodulator image that is suitable for displaying by the spatial lightmodulator, where noise introduced by the spatial light modulator isdistributed to regions outside the useful portion.

1G2. The method of 1G1 wherein assigning a phase value to each of theplurality of virtual-scene points to reduce speckle comprises assigningphase values to produce a smoothly interpolated version of a pluralityof points on a user's retina.

1G3. The method of 1G1 wherein assigning a phase value to each of theplurality of virtual-scene points to reduce speckle comprises assigningphase values to make optical paths from the virtual-scene points to theretina differ by integer multiples of a center wavelength of the lightsource.

1G4. The method of 1G1 wherein assigning a phase value to each of theplurality of virtual-scene points to reduce speckle comprises assigningphase values to make optical paths from the plurality of virtual-scenepoints to the pupil differ by integer multiples of a center wavelengthof the light source.

1G5. The method of 1G1 wherein determining a two-dimensionalcomplex-valued profile of a virtual-scene wave on a useful portion of anexit pupil plane comprises adding a spherical wave term for each of theplurality of virtual-scene points.

1G6. The method of 1G1 wherein back-propagating comprises incorporatingwave optics models of any optical components between the spatial lightmodulator and the useful portion of the exit pupil plane.

1G7. The method of 1G1 wherein determining a two-dimensionalcomplex-valued profile of an illumination wave comprises performing waveoptics analysis for components within an illumination module.

1G8. The method of 1G1 wherein encoding comprises encoding as aphase-only mask.

2A1. An apparatus comprising:

a transparent substrate having a first face through which a coherentlight beam emanates;

a light-scattering apparatus embedded in the substrate that scatterslight away from the first face; and

a reflective optical element to reflect the light from the scatteringapparatus to the first face and create the coherent light beam.

2A2. An apparatus to create a coherent light beam comprising:

a transparent substrate having a face and an embedded light-scatteringapparatus;

a light guiding apparatus positioned within the substrate to receivelight from outside the substrate and guide the light to the embeddedlight-scattering apparatus; and

a reflective optical element to reflect light scattered by thescattering apparatus to the face to create the coherent light beam.

2A3. An apparatus that includes a near-to-eye display device comprising:

at least one point light source;

a transparent substrate having a first face through which a coherentlight beam emanates;

a light-scattering apparatus embedded in the substrate to receive lightfrom the at least one point light source and scatter light away from thefirst face;

a reflective optical element to reflect the light from the scatteringapparatus to the first face to create the coherent light beam; and

a spatial light modulator mounted on the near-to-eye display device andilluminated by the coherent light beam, wherein the spatial lightmodulator is not in an optical conjugate plane to a retina of a userusing the near-to-eye display device.

2A4. An apparatus in accordance with any of 2A1-2A27 wherein thecoherent light beam comprises a converging light beam.

2A5. An apparatus in accordance with any of 2A1-2A27 wherein thecoherent light beam comprises a diverging light beam.

2A6. An apparatus in accordance with any of 2A1-2A27 wherein thecoherent light beam comprises a collimated light beam.

2A7. An apparatus in accordance with any of 2A1-2A27 wherein thereflective optical element comprises a micromirror array.

2A8. An apparatus in accordance with any of 2A1-2A27 wherein thereflective optical element comprises a Fresnel mirror.

2A9. An apparatus in accordance with any of 2A1-2A27 wherein thereflective optical element comprises a freeform optical reflector.

2A10. An apparatus in accordance with any of 2A1-2A27 wherein thereflective optical element comprises a concave mirror.

2A11. An apparatus in accordance with any of 2A1-2A27 wherein thereflective optical element reflects light to create a converging beamthat converges in one dimension.

2A12. An apparatus in accordance with any of 2A1-2A27 wherein thereflective optical element reflects light to create a converging beamthat converges in two dimensions.

2A13. An apparatus in accordance with any of 2A1-2A27, furthercomprising a spatial light modulator coupled to the first face of thetransparent substrate.

2A14. An apparatus in accordance with any of 2A1-2A27, wherein thespatial light modulator is transmissive.

2A15. An apparatus in accordance with any of 2A1-2A27, wherein thespatial light modulator is reflective.

2A16. An apparatus in accordance with any of 2A1-2A27, furthercomprising:

a point light source; and

a light guide within the substrate to guide light from the point lightsource to the scattering apparatus.

2A17. An apparatus in accordance with any of 2A1-2A27, furthercomprising a point light source within the substrate to provide light tothe scattering apparatus.

2A18. An apparatus in accordance with any of 2A1-2A27, wherein the pointlight source comprises an organic light emitting diode (OLED).

2A19. An apparatus in accordance with any of 2A1-2A27, wherein the pointlight source comprises a red organic light emitting diode (OLED), agreen OLED, and a blue OLED.

2A20. An apparatus in accordance with any of 2A1-2A27, wherein the pointlight source comprises a fluorescent molecule.

2A21. An apparatus in accordance with 2A20, wherein the fluorescentmolecule comprises a quantum dot.

2A22. An apparatus in accordance with any of 2A1-2A27, wherein thereflective optical element is embedded in the substrate.

2A23. An apparatus in accordance with any of 2A1-2A27, wherein thereflective optical element is transreflective.

2A24. An apparatus in accordance with any of 2A1-2A27, furthercomprising a point light source to provide light to the light guidingapparatus.

2A25. An apparatus in accordance with any of 2A1-2A27, wherein the atleast one point light source comprises a red light source, a green lightsource, and a blue light source.

2A26. An apparatus in accordance with any of 2A1-2A27, furthercomprising a light guide within the transparent substrate to guide lightfrom the at least one point light source to the light-scatteringapparatus.

2A27. An apparatus in accordance with any of 2A1-2A27, wherein thenear-to-eye display device comprises a head-worn device.

2B1. An apparatus comprising:

a slab waveguide having an input end, an output end, and first andsecond surfaces parallel to each other to cause light to propagate fromthe input end to the output end by total internal reflection;

a wedge coupled to receive light from the output end of the slabwaveguide, the wedge having a first surface, and a slanted surfacenonparallel to the first surface of the wedge to form a continuouslydecreasing thickness to cause light to exit the wedge from the slantedsurface; and an optical component having a face parallel to the slantedsurface of the wedge, the optical component including a micromirrorarray to reflect light received through the face back through the wedge.

2B2. The apparatus of 2B1 wherein the first surface of the wedge isparallel to the first surface of the slab waveguide.

2B3. The apparatus of 2B1 further comprising a spatial light modulatorpositioned on the first surface of the slab waveguide to modulate thelight as it propagates by total internal reflection.

2B4. The apparatus of 2B1 further comprising a spatial light modulatorpositioned between the wedge and the micromirror array to modulate thelight after leaving the slanted surface.

2B5. The apparatus of 2B1 further comprising a camera for eye tracking.

2B6. The apparatus of 2B5 wherein the camera is positioned along theslab waveguide.

2B7. An apparatus comprising:

a slab waveguide having an input end and an output end, the output endbeing formed as a first wedge, the first wedge including a first slantedsurface through which light exits after propagating from the input endthrough total internal reflection; and

a compensating wedge that includes a micromirror array to reflect lightexiting the first wedge.

2B8. The apparatus of 2B7 wherein the compensating wedge includes asecond slanted surface parallel to the first slanted surface.

2B9. The apparatus of 2B7 further comprising a spatial light modulatorpositioned along the slab waveguide to modulate the light as itpropagates by total internal reflection.

2B10. The apparatus of 2B9 further comprising a point light source toprovide light to the input end.

2B11. The apparatus of 2B7 further comprising a spatial light modulatorpositioned between the first wedge and the micromirror array to modulatethe light after leaving the first slanted surface.

2B12. The apparatus of 2B7 further comprising a camera for eye tracking.

2B13. The apparatus of 2B12 wherein the camera is positioned along theslab waveguide.

2B14. A near-to-eye display device comprising:

a point light source;

a slab waveguide having an input end, an output end, and first andsecond surfaces parallel to each other to cause light received from thepoint light source to propagate from the input end to the output end bytotal internal reflection;

a wedge coupled to receive light from the output end of the slabwaveguide, the wedge having a first surface, and a slanted surfacenonparallel to the first surface of the wedge to form a continuouslydecreasing thickness to cause light to exit the wedge from the slantedsurface;

an optical component having a face parallel to the slanted surface ofthe wedge, the optical component including a micromirror array toreflect light received through the face back through the wedge to createa converging light beam; and

a spatial light modulator illuminated by the converging light beam,wherein the spatial light modulator is not in an optical conjugate planeto a retina of a user using the near-to-eye display device.

2B15. The near-to-eye display device of 2B14 further comprising aspatial light modulator positioned on the first surface of the slabwaveguide to modulate the light as it propagates by total internalreflection.

2B16. The near-to-eye display device of 2B14 further comprising aspatial light modulator positioned between the wedge and the micromirrorarray to modulate the light after leaving the slanted surface.

2B17. The near-to-eye display device of 2B14 further comprising a camerafor eye tracking.

2B18. The near-to-eye display device of 2B17 wherein the camera ispositioned along the slab waveguide.

2B19. The near-to-eye display device of 2B14 wherein the opticalcomponent comprises a compensating wedge that when combined with thewedge produces a uniform thickness.

2B20. The near-to-eye display device of 2B14 wherein the near-to-eyedisplay device comprises a head-worn device.

2C1. An apparatus comprising:

a slab waveguide having an input end, an output end, and first andsecond surfaces parallel to each other to cause light to propagate fromthe input end to the output end by total internal reflection;

a curved wedge coupled to receive light from the output end of the slabwaveguide, the curved wedge having a continuously decreasing thicknessto cause light to exit the wedge from one of two surfaces.

2C2. The apparatus of 2C1 further comprising a spatial light modulatorpositioned on the first surface of the slab waveguide to modulate thelight as it propagates by total internal reflection.

2C3. The apparatus of 2C1 further comprising a camera for eye tracking.

2C4. The apparatus of 2C3 wherein the camera is positioned along theslab waveguide.

2C5. An apparatus comprising:

a slab waveguide having an input end and an output end, and first andsecond surfaces parallel to each other to cause light to propagate fromthe input end to the output end by total internal reflection;

a curved wedge coupled to receive light from the output end of the slabwaveguide, the curved wedge having a continuously decreasing thicknessto cause light to exit the wedge from one of two surfaces; and

a compensating curved wedge that provides a uniform optical path lengthfor light passing through both the curved wedge and the compensatingcurved wedge.

2C6. The apparatus of 2C5 further comprising a spatial light modulatorpositioned along the slab waveguide to modulate the light as itpropagates by total internal reflection.

2C7. The apparatus of 2C5 further comprising a point light source toprovide light to the input end.

2C8. The apparatus of 2C5 further comprising a camera for eye tracking.

2C9. The apparatus of 2C8 wherein the camera is positioned along theslab waveguide.

2C10. A near-to-eye display device comprising:

a point light source;

a slab waveguide having an input end, an output end, and first andsecond surfaces parallel to each other to cause light received from thepoint light source to propagate from the input end to the output end bytotal internal reflection;

a curved wedge coupled to receive light from the output end of the slabwaveguide, the wedge having first and second surfaces oriented to form acontinuously decreasing thickness to cause light to exit the curvedwedge from one of the first and second surface and create a converginglight beam;

a spatial light modulator illuminated by the converging light beam,wherein the spatial light modulator is not in an optical conjugate planeto a retina of a user using the near-to-eye display device.

2C11. The near-to-eye display device of 2C10 further comprising acompensating curved wedge that provides a uniform optical path lengthfor light passing through both the curved wedge and the compensatingcurved wedge.

2C12. The near-to-eye display device of 2C10 further comprising a camerafor eye tracking.

2C13. The near-to-eye display device of 2C10 wherein the camera ispositioned along the slab waveguide.

2C14. The near-to-eye display device of 2C10 wherein the near-to-eyedisplay device comprises a head-worn device.

3A1. A near-to-eye display device comprising:

a movable platform that includes a plurality of light sources; and

a circuit to modulate the plurality of light sources and to synchronizethe modulation with motion of the movable platform.

3A2. The near-to-eye display device of 3A1 further comprising apolarizing film to pass environmental light polarized in a firstorientation, wherein the plurality of light sources are positioned todirect light toward an expected location of a user's eye.

3A3. The near-to-eye display device of 3A1 wherein the plurality oflight sources are positioned to direct light away from an expectedlocation of a user's eye.

3A4. The near-to-eye display device of 3A1 wherein the plurality oflight sources includes an array of light sources.

3A5. The near-to-eye display device of 3A4 wherein the array of lightsources comprises an array of light emitting diodes.

3A6. The near-to-eye display device of 3A4 wherein the array of lightsources comprises light sources of at least two different colors.

3A7. The near-to-eye display device of 3A4 wherein the array of lightsources comprises red, green, and blue light sources.

3A8. The near-to-eye display device of 3A4 wherein the array of lightsources comprises a one-dimensional array.

3A9. The near-to-eye display device of 3A4 wherein the array of lightsources comprises a two-dimensional array.

3A10. The near-to-eye display device of 3A4 wherein the movable platformcomprises a bar that moves in one dimension.

3A11. The near-to-eye display device of 3A1 wherein the movable platformcomprises a bar mounted on a pivot point.

3A12. The near-to-eye display device of 3A1 wherein the movable platformcomprises a plurality of bars that move in one dimension.

3A13. The near-to-eye display device of 3A1 wherein the near-to-eyedisplay device comprises a head-worn device.

3A14. In combination:

a near-to-eye display device that comprises a movable platform thatincludes a plurality of light sources; and

a contact lens having a first portion and a second portion, the firstportion having a high-diopter lens to allow a user to focus at a planeof the plurality of light sources.

3A15. The combination of 3A14 wherein the near-to-eye display devicecomprises a head-worn device.

3A16. The combination of 3A14 wherein the near-to-eye display devicefurther comprises a polarizing film that polarizes light in a firstorientation, the polarizing film being oriented such that environmentallight viewed by a user of the near-to-eye display device passes throughthe polarizing film, and further oriented such that light produced bythe plurality of light sources does not pass through the polarizingfilm.

3A17. The combination of 3A16 wherein the plurality of light sourcesproduce light polarized in a second orientation different from the firstorientation.

3A18. The combination of 3A14 wherein the first portion of the contactlens has a polarization matching the polarizing film and the secondportion has a polarization matching the light produced by the pluralityof light sources.

3A19. The combination of 3A14 wherein the second portion of the contactlens includes color filtering.

3A20. The combination of 3A14 wherein the first and second portions ofthe contact lens are concentric.

3A21. The combination of 3A14 wherein the plurality of light sources ison a movable platform that sweeps over a viewing area of the near-to-eyedisplay device.

3A22. The combination of 3A21 wherein the movable platform comprises aplurality of bars that move in one dimension.

3A23. The combination of 3A21 wherein the movable platform moves in onedimension.

3A24. The combination of 3A21 wherein the movable platform is mounted tothe near-to-eye display device at a pivot point.

3A25. The combination of 3A14 wherein the second portion of the contactlens includes two color filters.

3A26. The combination of 3A14 wherein the second portion of the contactlens includes three color filters.

3A27. The combination of 3A14 wherein the second portion of the contactlens includes four color filters.

3A28. The combination of 3A14 wherein the movable platform ismagnetically actuated.

3A29. The combination of 3A14 wherein the movable platform ispiezoelectrically actuated.

3A30. The combination of 3A14 wherein the movable platform iselectrically actuated.

3A31. A near-to-eye display device comprising:

a point light source; and

a movable platform that includes a spatial light modulator positioned tobe illuminated by the point light source such that when the movableplatform is swept through a user's field of view, the spatial lightmodulator projects light on an exit pupil positioned at an expectedlocation of the user's eye pupil when the near-to-eye display device isin use, and wherein the exit pupil plane is at an optical conjugatelocation of the point light source.

3A32. The near-to-eye display device of 3A31 wherein the spatial lightmodulator includes a single row of pixels.

3A33. The near-to-eye display device of 3A31 wherein the spatial lightmodulator includes multiple rows of pixels.

3A34. The near-to-eye display device of 3A31 wherein the point lightsource is mounted on the moving platform.

3A35. The near-to-eye display device of 3A31 wherein the movableplatform comprises a plurality of bars that move in one dimension.

3A36. The near-to-eye display device of 3A31 wherein the moving platformcomprises a plurality of bars that each includes at least one row ofspatial light modulator pixels.

3A37. The near-to-eye display device of 3A31 further comprising aplurality of light sources of different colors that are time multiplexedwhen in use.

3A38. The near-to-eye display device of 3A31 wherein the near-to-eyedisplay device comprises a head-worn device.

4A1. A near-to-eye display device comprising:

a point light source;

a spatial light modulator;

a reflective optical element rotatably mounted to the near-to-eyedisplay device and positioned to be illuminated by the at least onepoint light source to project light on an exit pupil plane positioned atan expected location of a user's eye pupil when the near-to-eye displaydevice is in use;

a pupil-tracking device to determine a position of the user's eye pupil;and

an actuator to rotate the reflective optical element in response to theposition of the user's eye pupil.

4A2. The near-to-eye display device of 4A1 wherein the spatial lightmodulator is reflective.

4A3. The near-to-eye display device of 4A1 wherein the spatial lightmodulator is transmissive.

4A4. The near-to-eye display device of 4A1 wherein the spatial lightmodulator is coupled to the reflective optical element such that thespatial light modulator and the reflective optical element rotatetogether.

4A5. The near-to-eye display device of 4A1 wherein the near-to-eyedisplay device comprises a head-worn device.

4A6. The near-to-eye display device of 4A1 wherein the point lightsource is mounted on a nose bridge of the near-to-eye display device.

4A7. The near-to-eye display device of 4A1 wherein the point lightsource is mounted on a frame of the near-to-eye display device.

4A8. The near-to-eye display device of 4A1 wherein the actuatorcomprises a magnetic actuator.

4A9. The near-to-eye display device of 4A1 wherein the actuatorcomprises a motor.

4A10. The near-to-eye display device of 4A1 wherein light projected ontothe exit pupil plane includes multiple diffraction orders produced bythe spatial light modulator, and the actuator causes one of thediffraction orders to follow the position of the user's eye pupil.

4A11. The near-to-eye display device of 4A1, wherein light projectedonto the exit pupil plane includes multiple diffraction orders producedby the spatial light modulator, and the actuator causes a differentdiffraction order to follow the position of the user's eye pupil as theposition changes.

4A12. The near-to-eye display device of 4A1 further comprising aplurality of point light sources and a light selection componentresponsive to the pupil-tracking device.

4A13. A method comprising:

tracking the location of a user's eye pupil;

rotating a spatial light modulator that produces multiple diffractionorders so that a single diffraction order enters the user's eye pupil.

4A14. The method of 4A13 wherein tracking comprises measuring an angle,and further comprising driving the spatial light modulator withdifferent data to change the diffraction order that enters the user'seye pupil for angles above a threshold.

4A15. The method of 4A13 wherein tracking comprises measuring an angle,and further comprising selecting a different light source to illuminatethe spatial light modulator based on the angle.

4B1. A near-to-eye display device comprising:

a point light source;

a spatial light modulator;

an active grating that implements a multi-section prism disposed betweenthe point light source and the spatial light modulator, the activegrating being positioned to be illuminated by the point light source todirect light on the spatial light modulator, the spatial light modulatorbeing positioned to be illuminated by the wave directed by the activegrating to direct light on an exit pupil plane positioned at an expectedlocation of a user's eye pupil when the near-to-eye display device is inuse;

a pupil-tracking device to determine a position of the user's eye pupil;and

a control circuit to energize the active grating in response to theposition of the user's eye pupil.

4B2. A near-to-eye display device comprising:

a point light source;

an optical component that includes a spatial light modulator, areflector, and an active grating disposed between the reflector and thespatial light modulator, the optical component being positioned to beilluminated by the at least one point light source to project light onan exit pupil plane positioned at an expected location of a user's eyepupil when the near-to-eye display device is in use;

a pupil-tracking device to determine a position of the user's eye pupil;and

a control circuit to energize the active grating in response to theposition of the user's eye pupil.

4B3. A near-to-eye display device of any of 4B1-4B2 wherein thenear-to-eye display device comprises a head-worn device.

4B4. A method comprising:

tracking the location of a user's eye pupil;

actuating a programmable diffraction grating that steers light to aspatial light modulator that produces multiple diffraction orders sothat a single diffraction order enters the user's eye pupil.

4B5. The method of 4B4 wherein tracking comprises measuring an angle,and further comprising driving the spatial light modulator withdifferent data to change the diffraction order that enters the user'seye pupil for angles above a threshold.

4B6. The method of 4B4 wherein tracking comprises measuring an angle,and further comprising selecting a different light source to illuminatethe spatial light modulator based on the angle.

Although the present invention has been described in conjunction withcertain embodiments, it is to be understood that modifications andvariations may be resorted to without departing from the scope of theinvention as those skilled in the art readily understand. Suchmodifications and variations are considered to be within the scope ofthe invention and the appended claims.

What is claimed is:
 1. An apparatus comprising: a point light source that provides light; a light scattering structure that scatters light from the point light source; a light-guiding structure positioned to receive light from the point-light source and guide the light to the light-scattering structure; and a reflective optical element configured to reflect the light from the light-scattering structure and to create a coherent light beam; and a light-blocking unit that blocks a portion of the scattered light from the light-scattering structure.
 2. The apparatus of claim 1, wherein the coherent light beam comprises a converging light beam.
 3. The apparatus of claim 1, wherein the coherent light beam comprises a collimated light beam.
 4. The apparatus of claim 1, wherein the reflective optical element comprises a micromirror array.
 5. The apparatus of claim 1, wherein the reflective optical element comprises a Fresnel mirror.
 6. The apparatus of claim 1, wherein the reflective optical element comprises a freeform optical reflector.
 7. The apparatus of claim 1, wherein the reflective optical element comprises a concave mirror.
 8. The apparatus of claim 1, wherein the light-blocking unit is part of the light-scattering structure.
 9. The apparatus of claim 1, wherein the point light source includes a plurality of wavelengths selected from ultraviolet (UV), red, green, blue, and infrared (IR) light sources.
 10. The apparatus of claim 9, wherein the point light source comprises an organic light emitting diode (OLED).
 11. The apparatus of claim 9, wherein the point light source comprises a red organic light emitting diode (OLED), a green OLED, and a blue OLED.
 12. A near-to-eye display device, wherein the near-to-eye display device when in use is less than 25 cm from a user's eye, the near-to-eye display device comprising: a point light source that provides light; a light scattering structure that scatters light from the point light source; a light-guiding structure positioned to receive light from the point-light source and guide the light to the light-scattering structure; a reflective optical element configured to reflect the light from the light-scattering structure and to create a coherent light beam; a light-blocking unit that blocks a portion of the coherent light beam; a spatial light modulator mounted on the near-to-eye display device, wherein the light scattering apparatus receives light from the light-guiding structure, wherein the spatial light modulator forms a spatially coherent diverging light beam having a computer-generated hologram pattern, and wherein the light-blocking unit of the light scattering apparatus blocks at least part of the scattered light from the light-scattering structure; and a reflective optical element, wherein the reflective optical element receives the spatially coherent diverging light beam, wherein the reflective optical element has a positive optical power and forms a reflected coherent beam, and wherein the spatial light modulator receives the reflected spatially coherent beam.
 13. The near-to-eye display device of claim 12, wherein the scattering apparatus comprises transparent nanoparticles.
 14. The near-to-eye display device of claim 12, wherein the scattering apparatus comprises microparticles.
 15. The near-to-eye display device of claim 12, wherein the light-blocking unit is a mirror.
 16. The near-to-eye display device of claim 12, wherein the light-blocking unit is composed of a light absorbing material.
 17. The near-to-eye display device of claim 12, wherein the light scattering apparatus and the reflective optical element are embedded in a transparent substrate.
 18. The near-to-eye display device of claim 12, a transparent substrate having a first face through which the coherent light beam emanates, wherein the light scattering apparatus is in the substrate.
 19. The near-to-eye display device of claim 12, wherein the spatial light modulator is transmissive.
 20. A method comprising: mounting a spatial light modulator (SLM) on a near-to-eye display device, wherein the near-to-eye display device when in use is less than 25 cm from a user's eye; guiding light from a point light source and scattering the guided light; reflecting the scattered light to create a coherent light beam; blocking a portion of the coherent light beam to form a partially blocked coherent beam; and modulating the partially blocked coherent beam with the SLM. 